/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 Aug 27 23:45:47.200: Unexpected error: <*errors.errorString | 0xc0002801f0>: { s: "timed out waiting for the condition", } timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/pods.go:103 [BeforeEach] [k8s.io] Pods /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client Aug 27 23:40:47.155: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename pods STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [k8s.io] Pods /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:181 [It] should support remote command execution over websockets [NodeConformance] [Conformance] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 Aug 27 23:40:47.174: INFO: >>> kubeConfig: /root/.kube/config STEP: creating the pod STEP: submitting the pod to kubernetes Aug 27 23:45:47.199: FAIL: Unexpected error: <*errors.errorString | 0xc0002801f0>: { s: "timed out waiting for the condition", } timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/framework.(*PodClient).CreateSync(0xc00434c2c0, 0xc0060a1c00, 0x0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/pods.go:103 +0xfe k8s.io/kubernetes/test/e2e/common.glob..func18.7() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:560 +0x48c k8s.io/kubernetes/test/e2e.RunE2ETests(0xc00180a780) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc00180a780) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc00180a780, 0x4dec428) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [k8s.io] Pods /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "pods-1698". STEP: Found 7 events. Aug 27 23:45:47.204: INFO: At 2021-08-27 23:40:47 +0000 UTC - event for pod-exec-websocket-54601d43-644c-422b-9077-c973495ab1ed: {default-scheduler } Scheduled: Successfully assigned pods-1698/pod-exec-websocket-54601d43-644c-422b-9077-c973495ab1ed to node2 Aug 27 23:45:47.204: INFO: At 2021-08-27 23:40:49 +0000 UTC - event for pod-exec-websocket-54601d43-644c-422b-9077-c973495ab1ed: {multus } AddedInterface: Add eth0 [10.244.4.11/24] Aug 27 23:45:47.204: INFO: At 2021-08-27 23:40:50 +0000 UTC - event for pod-exec-websocket-54601d43-644c-422b-9077-c973495ab1ed: {kubelet node2} Pulling: Pulling image "docker.io/library/busybox:1.29" Aug 27 23:45:47.204: INFO: At 2021-08-27 23:40:50 +0000 UTC - event for pod-exec-websocket-54601d43-644c-422b-9077-c973495ab1ed: {kubelet node2} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit Aug 27 23:45:47.204: INFO: At 2021-08-27 23:40:50 +0000 UTC - event for pod-exec-websocket-54601d43-644c-422b-9077-c973495ab1ed: {kubelet node2} Failed: Error: ErrImagePull Aug 27 23:45:47.204: INFO: At 2021-08-27 23:40:51 +0000 UTC - event for pod-exec-websocket-54601d43-644c-422b-9077-c973495ab1ed: {kubelet node2} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" Aug 27 23:45:47.204: INFO: At 2021-08-27 23:40:51 +0000 UTC - event for pod-exec-websocket-54601d43-644c-422b-9077-c973495ab1ed: {kubelet node2} Failed: Error: ImagePullBackOff Aug 27 23:45:47.207: INFO: POD NODE PHASE GRACE CONDITIONS Aug 27 23:45:47.207: INFO: pod-exec-websocket-54601d43-644c-422b-9077-c973495ab1ed node2 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-08-27 23:40:47 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-08-27 23:40:47 +0000 UTC ContainersNotReady containers with unready status: [main]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-08-27 23:40:47 +0000 UTC ContainersNotReady containers with unready status: [main]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-08-27 23:40:47 +0000 UTC }] Aug 27 23:45:47.207: INFO: Aug 27 23:45:47.211: INFO: Logging node info for node master1 Aug 27 23:45:47.214: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 3af53387-5aee-42c1-b0e7-644cf9161d48 58574 0 2021-08-27 20:46:13 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"92:f8:b6:72:e4:be"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-08-27 20:46:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-08-27 20:46:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {kube-controller-manager Update v1 2021-08-27 20:46:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:50:56 +0000 UTC,LastTransitionTime:2021-08-27 20:50:56 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-27 23:45:39 +0000 UTC,LastTransitionTime:2021-08-27 20:46:13 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-27 23:45:39 +0000 UTC,LastTransitionTime:2021-08-27 20:46:13 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-27 23:45:39 +0000 UTC,LastTransitionTime:2021-08-27 20:46:13 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-27 23:45:39 +0000 UTC,LastTransitionTime:2021-08-27 20:50:30 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:5d584023135a46ecb77596bf48ed7f2f,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:587cafa0-6de3-49f8-906e-06315a8ff104,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[sirot/netperf-latest@sha256:23929b922bb077cb341450fc1c90ae42e6f75da5d7ce61cd1880b08560a5ff85 sirot/netperf-latest:latest],SizeBytes:282025213,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:ae72171f047a37ee5423e0692df7429830919af16e9d668ab0c80b723863d102 tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:cd98d1edca8e5e2e3ea42cbc463812483e5d069d10f0974ca9d484b5a7bd68db tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[aquasec/kube-bench@sha256:3544f6662feb73d36fdba35b17652e2fd73aae45bd4b60e76d7ab928220b3cc6 aquasec/kube-bench:0.3.1],SizeBytes:19301876,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Aug 27 23:45:47.215: INFO: Logging kubelet events for node master1 Aug 27 23:45:47.218: INFO: Logging pods the kubelet thinks is on node master1 Aug 27 23:45:47.228: INFO: kube-multus-ds-amd64-sfr9k started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.228: INFO: Container kube-multus ready: true, restart count 1 Aug 27 23:45:47.228: INFO: docker-registry-docker-registry-56cbc7bc58-cthtt started at 2021-08-27 20:51:47 +0000 UTC (0+2 container statuses recorded) Aug 27 23:45:47.228: INFO: Container docker-registry ready: true, restart count 0 Aug 27 23:45:47.228: INFO: Container nginx ready: true, restart count 0 Aug 27 23:45:47.228: INFO: kube-scheduler-master1 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.228: INFO: Container kube-scheduler ready: true, restart count 0 Aug 27 23:45:47.229: INFO: kube-flannel-pp7vp started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded) Aug 27 23:45:47.229: INFO: Init container install-cni ready: true, restart count 0 Aug 27 23:45:47.229: INFO: Container kube-flannel ready: true, restart count 3 Aug 27 23:45:47.229: INFO: kube-proxy-rb5p6 started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.229: INFO: Container kube-proxy ready: true, restart count 1 Aug 27 23:45:47.229: INFO: coredns-7677f9bb54-dwtp5 started at 2021-08-27 20:49:16 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.229: INFO: Container coredns ready: true, restart count 1 Aug 27 23:45:47.229: INFO: node-exporter-z2ngr started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded) Aug 27 23:45:47.229: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 27 23:45:47.229: INFO: Container node-exporter ready: true, restart count 0 Aug 27 23:45:47.229: INFO: kube-apiserver-master1 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.229: INFO: Container kube-apiserver ready: true, restart count 0 Aug 27 23:45:47.229: INFO: kube-controller-manager-master1 started at 2021-08-27 20:54:27 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.229: INFO: Container kube-controller-manager ready: true, restart count 3 W0827 23:45:47.242257 26 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. Aug 27 23:45:47.273: INFO: Latency metrics for node master1 Aug 27 23:45:47.273: INFO: Logging node info for node master2 Aug 27 23:45:47.276: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 666e473f-d9e6-4c06-8b56-06474d788f70 58636 0 2021-08-27 20:46:51 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"42:86:ff:30:bd:4d"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-08-27 20:46:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {kubelet Update v1 2021-08-27 20:46:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-08-27 20:48:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-08-27 20:55:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:51:53 +0000 UTC,LastTransitionTime:2021-08-27 20:51:53 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-27 23:45:47 +0000 UTC,LastTransitionTime:2021-08-27 20:46:51 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-27 23:45:47 +0000 UTC,LastTransitionTime:2021-08-27 20:46:51 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-27 23:45:47 +0000 UTC,LastTransitionTime:2021-08-27 20:46:51 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-27 23:45:47 +0000 UTC,LastTransitionTime:2021-08-27 20:48:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:2835065974a64998811b9acd85de209b,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:309b9155-1e2a-4ebd-900f-bba5abfc3a5d,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[sirot/netperf-latest@sha256:23929b922bb077cb341450fc1c90ae42e6f75da5d7ce61cd1880b08560a5ff85 sirot/netperf-latest:latest],SizeBytes:282025213,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Aug 27 23:45:47.277: INFO: Logging kubelet events for node master2 Aug 27 23:45:47.278: INFO: Logging pods the kubelet thinks is on node master2 Aug 27 23:45:47.287: INFO: kube-apiserver-master2 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.287: INFO: Container kube-apiserver ready: true, restart count 0 Aug 27 23:45:47.287: INFO: kube-scheduler-master2 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.287: INFO: Container kube-scheduler ready: true, restart count 2 Aug 27 23:45:47.287: INFO: kube-proxy-b4mn9 started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.287: INFO: Container kube-proxy ready: true, restart count 1 Aug 27 23:45:47.287: INFO: node-feature-discovery-controller-5bf5c49849-zr9zd started at 2021-08-27 20:55:09 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.287: INFO: Container nfd-controller ready: true, restart count 0 Aug 27 23:45:47.287: INFO: kube-controller-manager-master2 started at 2021-08-27 20:51:01 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.287: INFO: Container kube-controller-manager ready: true, restart count 2 Aug 27 23:45:47.287: INFO: kube-flannel-4znnq started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded) Aug 27 23:45:47.287: INFO: Init container install-cni ready: true, restart count 2 Aug 27 23:45:47.287: INFO: Container kube-flannel ready: true, restart count 2 Aug 27 23:45:47.287: INFO: kube-multus-ds-amd64-4mgbk started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.287: INFO: Container kube-multus ready: true, restart count 1 Aug 27 23:45:47.287: INFO: prometheus-operator-5bb8cb9d8f-whr5p started at 2021-08-27 20:59:06 +0000 UTC (0+2 container statuses recorded) Aug 27 23:45:47.287: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 27 23:45:47.287: INFO: Container prometheus-operator ready: true, restart count 0 Aug 27 23:45:47.287: INFO: node-exporter-96jk5 started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded) Aug 27 23:45:47.287: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 27 23:45:47.287: INFO: Container node-exporter ready: true, restart count 0 W0827 23:45:47.303897 26 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. Aug 27 23:45:47.331: INFO: Latency metrics for node master2 Aug 27 23:45:47.331: INFO: Logging node info for node master3 Aug 27 23:45:47.335: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 620b6dba-f2c5-46e9-b2ff-d2f4197167d0 58630 0 2021-08-27 20:47:02 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ce:de:ea:c3:40:4f"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-08-27 20:47:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-08-27 20:47:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-08-27 20:48:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:51:44 +0000 UTC,LastTransitionTime:2021-08-27 20:51:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-27 23:45:45 +0000 UTC,LastTransitionTime:2021-08-27 20:47:02 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-27 23:45:45 +0000 UTC,LastTransitionTime:2021-08-27 20:47:02 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-27 23:45:45 +0000 UTC,LastTransitionTime:2021-08-27 20:47:02 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-27 23:45:45 +0000 UTC,LastTransitionTime:2021-08-27 20:48:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:37decbffe0e84048b5801289ad3be5bf,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:c96fe8b9-1ce0-44cd-935a-b58987e26570,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496891,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Aug 27 23:45:47.335: INFO: Logging kubelet events for node master3 Aug 27 23:45:47.338: INFO: Logging pods the kubelet thinks is on node master3 Aug 27 23:45:47.347: INFO: kube-flannel-fkz5d started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded) Aug 27 23:45:47.347: INFO: Init container install-cni ready: true, restart count 0 Aug 27 23:45:47.347: INFO: Container kube-flannel ready: true, restart count 1 Aug 27 23:45:47.347: INFO: dns-autoscaler-5b7b5c9b6f-54xch started at 2021-08-27 20:49:19 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.347: INFO: Container autoscaler ready: true, restart count 1 Aug 27 23:45:47.347: INFO: coredns-7677f9bb54-rxplt started at 2021-08-27 20:49:21 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.347: INFO: Container coredns ready: true, restart count 1 Aug 27 23:45:47.347: INFO: kube-controller-manager-master3 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.347: INFO: Container kube-controller-manager ready: true, restart count 2 Aug 27 23:45:47.347: INFO: kube-proxy-8sxhm started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.347: INFO: Container kube-proxy ready: true, restart count 1 Aug 27 23:45:47.347: INFO: kube-multus-ds-amd64-wwcgv started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.347: INFO: Container kube-multus ready: true, restart count 1 Aug 27 23:45:47.347: INFO: node-exporter-d4m7q started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded) Aug 27 23:45:47.347: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 27 23:45:47.347: INFO: Container node-exporter ready: true, restart count 0 Aug 27 23:45:47.347: INFO: kube-apiserver-master3 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.347: INFO: Container kube-apiserver ready: true, restart count 0 Aug 27 23:45:47.347: INFO: kube-scheduler-master3 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.347: INFO: Container kube-scheduler ready: true, restart count 2 W0827 23:45:47.362877 26 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. Aug 27 23:45:47.388: INFO: Latency metrics for node master3 Aug 27 23:45:47.388: INFO: Logging node info for node node1 Aug 27 23:45:47.391: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 e7a2481e-32db-4c83-bd9f-4a0687258e7a 58635 0 2021-08-27 20:48:09 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.36.2.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"56:c7:37:40:51:ca"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-08-27 20:48:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubeadm Update v1 2021-08-27 20:48:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-08-27 20:55:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-08-27 20:57:40 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-08-27 20:57:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269628928 0} {<nil>} 196552372Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884628480 0} {<nil>} 174692020Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:51:49 +0000 UTC,LastTransitionTime:2021-08-27 20:51:49 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-27 23:45:47 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-27 23:45:47 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-27 23:45:47 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-27 23:45:47 +0000 UTC,LastTransitionTime:2021-08-27 20:48:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:c1e38e80ea114a5f96601202301ce842,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:e769e86d-15c0-442c-a93b-bcc6c33ff1cd,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:331c6faa8b0d5ec72cf105e87d35df0a2f2baeec3d6217a51faa73f9460f937f localhost:30500/barometer-collectd:stable],SizeBytes:1238704157,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:a7cea43d9d2f67c38fbf0407786edbe660ee9072945f7bb272b55fd255e8eaca opnfv/barometer-collectd:stable],SizeBytes:1075746799,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:1003787960,},ContainerImage{Names:[localhost:30500/cmk@sha256:fd1487b0c07556a087eff669e70c501a704720dcd53ff75183593de6720585f2 cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[aquasec/kube-hunter@sha256:2be6820bc1d7e0f57193a9a27d5a3e16b2fd93c53747b03ce8ca48c6fc323781 aquasec/kube-hunter:0.3.1],SizeBytes:347611549,},ContainerImage{Names:[golang@sha256:db2475a1dbb2149508e5db31d7d77a75e6600d54be645f37681f03f2762169ba golang:alpine3.12],SizeBytes:301186719,},ContainerImage{Names:[sirot/netperf-latest@sha256:23929b922bb077cb341450fc1c90ae42e6f75da5d7ce61cd1880b08560a5ff85 sirot/netperf-latest:latest],SizeBytes:282025213,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:d7300ccf7ff3e9cea2111d275143b8050618bbc1d1ffe41f46286b1696261243 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44393508,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:de25c7fc6c4f3a27c7f0c2dff454e4671823a34d88abd533f210848d527e0fbb alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Aug 27 23:45:47.392: INFO: Logging kubelet events for node node1 Aug 27 23:45:47.395: INFO: Logging pods the kubelet thinks is on node node1 Aug 27 23:45:47.410: INFO: kubernetes-dashboard-86c6f9df5b-c56fg started at 2021-08-27 20:49:21 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.410: INFO: Container kubernetes-dashboard ready: true, restart count 1 Aug 27 23:45:47.410: INFO: node-feature-discovery-worker-bd9kg started at 2021-08-27 20:55:06 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.410: INFO: Container nfd-worker ready: true, restart count 0 Aug 27 23:45:47.410: INFO: netserver-0 started at 2021-08-27 23:45:39 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.410: INFO: Container webserver ready: false, restart count 0 Aug 27 23:45:47.410: INFO: kube-proxy-pb5bl started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.410: INFO: Container kube-proxy ready: true, restart count 2 Aug 27 23:45:47.410: INFO: collectd-ccvwg started at 2021-08-27 21:04:15 +0000 UTC (0+3 container statuses recorded) Aug 27 23:45:47.410: INFO: Container collectd ready: true, restart count 0 Aug 27 23:45:47.410: INFO: Container collectd-exporter ready: true, restart count 0 Aug 27 23:45:47.410: INFO: Container rbac-proxy ready: true, restart count 0 Aug 27 23:45:47.410: INFO: ss2-1 started at 2021-08-27 23:39:37 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.410: INFO: Container webserver ready: true, restart count 0 Aug 27 23:45:47.410: INFO: prometheus-k8s-0 started at 2021-08-27 20:59:29 +0000 UTC (0+5 container statuses recorded) Aug 27 23:45:47.410: INFO: Container custom-metrics-apiserver ready: true, restart count 0 Aug 27 23:45:47.410: INFO: Container grafana ready: true, restart count 0 Aug 27 23:45:47.410: INFO: Container prometheus ready: true, restart count 1 Aug 27 23:45:47.410: INFO: Container prometheus-config-reloader ready: true, restart count 0 Aug 27 23:45:47.410: INFO: Container rules-configmap-reloader ready: true, restart count 0 Aug 27 23:45:47.410: INFO: pod-secrets-5182bdb2-d9a2-4230-8371-839302aee1c7 started at 2021-08-27 23:40:51 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.410: INFO: Container secret-env-test ready: false, restart count 0 Aug 27 23:45:47.410: INFO: kube-flannel-ssxn7 started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded) Aug 27 23:45:47.410: INFO: Init container install-cni ready: true, restart count 0 Aug 27 23:45:47.410: INFO: Container kube-flannel ready: true, restart count 1 Aug 27 23:45:47.410: INFO: pod-qos-class-84c512ab-e22b-4ee0-a2e2-29af708b4de0 started at 2021-08-27 23:45:13 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.410: INFO: Container agnhost ready: true, restart count 0 Aug 27 23:45:47.410: INFO: kube-multus-ds-amd64-nn7bl started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.410: INFO: Container kube-multus ready: true, restart count 1 Aug 27 23:45:47.410: INFO: kubernetes-metrics-scraper-678c97765c-gtp5x started at 2021-08-27 20:49:21 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.410: INFO: Container kubernetes-metrics-scraper ready: true, restart count 1 Aug 27 23:45:47.410: INFO: cmk-init-discover-node1-spg26 started at 2021-08-27 20:57:37 +0000 UTC (0+3 container statuses recorded) Aug 27 23:45:47.410: INFO: Container discover ready: false, restart count 0 Aug 27 23:45:47.410: INFO: Container init ready: false, restart count 0 Aug 27 23:45:47.410: INFO: Container install ready: false, restart count 0 Aug 27 23:45:47.410: INFO: cmk-jw4m6 started at 2021-08-27 20:58:19 +0000 UTC (0+2 container statuses recorded) Aug 27 23:45:47.410: INFO: Container nodereport ready: true, restart count 0 Aug 27 23:45:47.410: INFO: Container reconcile ready: true, restart count 0 Aug 27 23:45:47.410: INFO: nginx-proxy-node1 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.410: INFO: Container nginx-proxy ready: true, restart count 2 Aug 27 23:45:47.410: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-9lndx started at 2021-08-27 20:55:51 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.410: INFO: Container kube-sriovdp ready: true, restart count 0 Aug 27 23:45:47.410: INFO: node-exporter-4cvlq started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded) Aug 27 23:45:47.410: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 27 23:45:47.410: INFO: Container node-exporter ready: true, restart count 0 Aug 27 23:45:47.410: INFO: var-expansion-5cb4d930-ff54-47be-a198-6adb16e9fa01 started at 2021-08-27 23:41:36 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.410: INFO: Container dapi-container ready: false, restart count 0 W0827 23:45:47.425878 26 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. Aug 27 23:45:47.471: INFO: Latency metrics for node node1 Aug 27 23:45:47.471: INFO: Logging node info for node node2 Aug 27 23:45:47.473: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 eb7da7c0-513f-4072-a078-ad3d24f88114 58637 0 2021-08-27 20:48:09 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.36.2.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"9a:88:9c:d1:39:68"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-08-27 20:48:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubeadm Update v1 2021-08-27 20:48:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-08-27 20:55:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-08-27 20:58:03 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-08-27 20:58:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269628928 0} {<nil>} 196552372Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884628480 0} {<nil>} 174692020Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:50:36 +0000 UTC,LastTransitionTime:2021-08-27 20:50:36 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-27 23:45:47 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-27 23:45:47 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-27 23:45:47 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-27 23:45:47 +0000 UTC,LastTransitionTime:2021-08-27 20:48:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:0ccfc2a4a9b7400c9ca53b5de0ca4970,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:26b02690-0814-4c92-9f6d-d315df796ce6,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:331c6faa8b0d5ec72cf105e87d35df0a2f2baeec3d6217a51faa73f9460f937f localhost:30500/barometer-collectd:stable],SizeBytes:1238704157,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[localhost:30500/cmk@sha256:fd1487b0c07556a087eff669e70c501a704720dcd53ff75183593de6720585f2 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:d7300ccf7ff3e9cea2111d275143b8050618bbc1d1ffe41f46286b1696261243 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44393508,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:ae72171f047a37ee5423e0692df7429830919af16e9d668ab0c80b723863d102 localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:cd98d1edca8e5e2e3ea42cbc463812483e5d069d10f0974ca9d484b5a7bd68db localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[aquasec/kube-bench@sha256:3544f6662feb73d36fdba35b17652e2fd73aae45bd4b60e76d7ab928220b3cc6 aquasec/kube-bench:0.3.1],SizeBytes:19301876,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Aug 27 23:45:47.474: INFO: Logging kubelet events for node node2 Aug 27 23:45:47.477: INFO: Logging pods the kubelet thinks is on node node2 Aug 27 23:45:47.490: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-v99df started at 2021-08-27 21:02:08 +0000 UTC (0+2 container statuses recorded) Aug 27 23:45:47.490: INFO: Container tas-controller ready: true, restart count 0 Aug 27 23:45:47.490: INFO: Container tas-extender ready: true, restart count 0 Aug 27 23:45:47.490: INFO: ss2-2 started at 2021-08-27 23:40:26 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.490: INFO: Container webserver ready: false, restart count 0 Aug 27 23:45:47.490: INFO: pod-logs-websocket-f7bdb0b2-32b1-4320-8388-1166a6cbc9ab started at 2021-08-27 23:45:22 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.490: INFO: Container main ready: false, restart count 0 Aug 27 23:45:47.490: INFO: cmk-webhook-6c9d5f8578-ndbx2 started at 2021-08-27 20:58:20 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.490: INFO: Container cmk-webhook ready: true, restart count 0 Aug 27 23:45:47.490: INFO: cmk-init-discover-node2-l9qjd started at 2021-08-27 20:57:57 +0000 UTC (0+3 container statuses recorded) Aug 27 23:45:47.490: INFO: Container discover ready: false, restart count 0 Aug 27 23:45:47.490: INFO: Container init ready: false, restart count 0 Aug 27 23:45:47.490: INFO: Container install ready: false, restart count 0 Aug 27 23:45:47.490: INFO: kube-flannel-t9qv4 started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded) Aug 27 23:45:47.490: INFO: Init container install-cni ready: true, restart count 0 Aug 27 23:45:47.490: INFO: Container kube-flannel ready: true, restart count 2 Aug 27 23:45:47.490: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4f962 started at 2021-08-27 20:55:51 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.490: INFO: Container kube-sriovdp ready: true, restart count 0 Aug 27 23:45:47.490: INFO: node-feature-discovery-worker-54lfh started at 2021-08-27 20:55:06 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.490: INFO: Container nfd-worker ready: true, restart count 0 Aug 27 23:45:47.490: INFO: kube-proxy-r4q4t started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.490: INFO: Container kube-proxy ready: true, restart count 2 Aug 27 23:45:47.490: INFO: kube-multus-ds-amd64-tfffk started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.490: INFO: Container kube-multus ready: true, restart count 1 Aug 27 23:45:47.490: INFO: node-exporter-p6h5h started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded) Aug 27 23:45:47.490: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 27 23:45:47.490: INFO: Container node-exporter ready: true, restart count 0 Aug 27 23:45:47.490: INFO: nginx-proxy-node2 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.490: INFO: Container nginx-proxy ready: true, restart count 1 Aug 27 23:45:47.491: INFO: cmk-fzjgr started at 2021-08-27 20:58:20 +0000 UTC (0+2 container statuses recorded) Aug 27 23:45:47.491: INFO: Container nodereport ready: true, restart count 0 Aug 27 23:45:47.491: INFO: Container reconcile ready: true, restart count 0 Aug 27 23:45:47.491: INFO: netserver-1 started at 2021-08-27 23:45:39 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.491: INFO: Container webserver ready: false, restart count 0 Aug 27 23:45:47.491: INFO: ss2-0 started at 2021-08-27 23:39:25 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.491: INFO: Container webserver ready: true, restart count 0 Aug 27 23:45:47.491: INFO: pod-service-account-29901aef-ff16-49bc-ac41-0c970d5adaac started at 2021-08-27 23:43:33 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.491: INFO: Container test ready: false, restart count 0 Aug 27 23:45:47.491: INFO: var-expansion-fda203cd-93b6-4495-bddf-3b7b58556a19 started at 2021-08-27 23:45:03 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.491: INFO: Container dapi-container ready: false, restart count 0 Aug 27 23:45:47.491: INFO: collectd-64dp2 started at 2021-08-27 21:04:15 +0000 UTC (0+3 container statuses recorded) Aug 27 23:45:47.491: INFO: Container collectd ready: true, restart count 0 Aug 27 23:45:47.491: INFO: Container collectd-exporter ready: true, restart count 0 Aug 27 23:45:47.491: INFO: Container rbac-proxy ready: true, restart count 0 Aug 27 23:45:47.491: INFO: pod-exec-websocket-54601d43-644c-422b-9077-c973495ab1ed started at 2021-08-27 23:40:47 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.491: INFO: Container main ready: false, restart count 0 Aug 27 23:45:47.491: INFO: ss-0 started at 2021-08-27 23:45:13 +0000 UTC (0+1 container statuses recorded) Aug 27 23:45:47.491: INFO: Container webserver ready: false, restart count 0 W0827 23:45:47.506373 26 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. Aug 27 23:45:47.565: INFO: Latency metrics for node node2 Aug 27 23:45:47.565: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "pods-1698" for this suite. /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 Aug 27 23:50:47.741: starting pod busybox-d46f04c7-6f69-4518-8c43-12c7adffe2f5 in namespace container-probe-6247 Unexpected error: <*errors.errorString | 0xc0002801f0>: { s: "timed out waiting for the condition", } timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/container_probe.go:426 [BeforeEach] [k8s.io] Probing container /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client Aug 27 23:45:47.695: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename container-probe STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [k8s.io] Probing container /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/container_probe.go:54 [It] should be restarted with a exec "cat /tmp/health" liveness probe [NodeConformance] [Conformance] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 STEP: Creating pod busybox-d46f04c7-6f69-4518-8c43-12c7adffe2f5 in namespace container-probe-6247 Aug 27 23:50:47.740: FAIL: starting pod busybox-d46f04c7-6f69-4518-8c43-12c7adffe2f5 in namespace container-probe-6247 Unexpected error: <*errors.errorString | 0xc0002801f0>: { s: "timed out waiting for the condition", } timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/common.RunLivenessTest(0xc0001ade40, 0xc0028b1000, 0x1, 0x37e11d6000) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/container_probe.go:426 +0x4b9 k8s.io/kubernetes/test/e2e/common.glob..func3.4() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/container_probe.go:130 +0x197 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc00180a780) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc00180a780) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc00180a780, 0x4dec428) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 STEP: deleting the pod [AfterEach] [k8s.io] Probing container /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "container-probe-6247". STEP: Found 9 events. Aug 27 23:50:47.751: INFO: At 2021-08-27 23:45:47 +0000 UTC - event for busybox-d46f04c7-6f69-4518-8c43-12c7adffe2f5: {default-scheduler } Scheduled: Successfully assigned container-probe-6247/busybox-d46f04c7-6f69-4518-8c43-12c7adffe2f5 to node2 Aug 27 23:50:47.751: INFO: At 2021-08-27 23:45:49 +0000 UTC - event for busybox-d46f04c7-6f69-4518-8c43-12c7adffe2f5: {multus } AddedInterface: Add eth0 [10.244.4.41/24] Aug 27 23:50:47.751: INFO: At 2021-08-27 23:45:49 +0000 UTC - event for busybox-d46f04c7-6f69-4518-8c43-12c7adffe2f5: {kubelet node2} Pulling: Pulling image "docker.io/library/busybox:1.29" Aug 27 23:50:47.751: INFO: At 2021-08-27 23:45:50 +0000 UTC - event for busybox-d46f04c7-6f69-4518-8c43-12c7adffe2f5: {kubelet node2} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit Aug 27 23:50:47.751: INFO: At 2021-08-27 23:45:50 +0000 UTC - event for busybox-d46f04c7-6f69-4518-8c43-12c7adffe2f5: {kubelet node2} Failed: Error: ErrImagePull Aug 27 23:50:47.751: INFO: At 2021-08-27 23:45:50 +0000 UTC - event for busybox-d46f04c7-6f69-4518-8c43-12c7adffe2f5: {kubelet node2} SandboxChanged: Pod sandbox changed, it will be killed and re-created. Aug 27 23:50:47.751: INFO: At 2021-08-27 23:45:52 +0000 UTC - event for busybox-d46f04c7-6f69-4518-8c43-12c7adffe2f5: {multus } AddedInterface: Add eth0 [10.244.4.42/24] Aug 27 23:50:47.751: INFO: At 2021-08-27 23:45:52 +0000 UTC - event for busybox-d46f04c7-6f69-4518-8c43-12c7adffe2f5: {kubelet node2} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" Aug 27 23:50:47.751: INFO: At 2021-08-27 23:45:52 +0000 UTC - event for busybox-d46f04c7-6f69-4518-8c43-12c7adffe2f5: {kubelet node2} Failed: Error: ImagePullBackOff Aug 27 23:50:47.753: INFO: POD NODE PHASE GRACE CONDITIONS Aug 27 23:50:47.753: INFO: Aug 27 23:50:47.757: INFO: Logging node info for node master1 Aug 27 23:50:47.760: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 3af53387-5aee-42c1-b0e7-644cf9161d48 61126 0 2021-08-27 20:46:13 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"92:f8:b6:72:e4:be"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-08-27 20:46:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-08-27 20:46:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {kube-controller-manager Update v1 2021-08-27 20:46:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:50:56 +0000 UTC,LastTransitionTime:2021-08-27 20:50:56 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-27 23:50:40 +0000 UTC,LastTransitionTime:2021-08-27 20:46:13 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-27 23:50:40 +0000 UTC,LastTransitionTime:2021-08-27 20:46:13 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-27 23:50:40 +0000 UTC,LastTransitionTime:2021-08-27 20:46:13 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-27 23:50:40 +0000 UTC,LastTransitionTime:2021-08-27 20:50:30 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:5d584023135a46ecb77596bf48ed7f2f,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:587cafa0-6de3-49f8-906e-06315a8ff104,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[sirot/netperf-latest@sha256:23929b922bb077cb341450fc1c90ae42e6f75da5d7ce61cd1880b08560a5ff85 sirot/netperf-latest:latest],SizeBytes:282025213,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:ae72171f047a37ee5423e0692df7429830919af16e9d668ab0c80b723863d102 tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:cd98d1edca8e5e2e3ea42cbc463812483e5d069d10f0974ca9d484b5a7bd68db tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[aquasec/kube-bench@sha256:3544f6662feb73d36fdba35b17652e2fd73aae45bd4b60e76d7ab928220b3cc6 aquasec/kube-bench:0.3.1],SizeBytes:19301876,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Aug 27 23:50:47.761: INFO: Logging kubelet events for node master1 Aug 27 23:50:47.763: INFO: Logging pods the kubelet thinks is on node master1 Aug 27 23:50:47.771: INFO: kube-apiserver-master1 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.771: INFO: Container kube-apiserver ready: true, restart count 0 Aug 27 23:50:47.771: INFO: kube-controller-manager-master1 started at 2021-08-27 20:54:27 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.771: INFO: Container kube-controller-manager ready: true, restart count 3 Aug 27 23:50:47.771: INFO: kube-proxy-rb5p6 started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.771: INFO: Container kube-proxy ready: true, restart count 1 Aug 27 23:50:47.771: INFO: coredns-7677f9bb54-dwtp5 started at 2021-08-27 20:49:16 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.771: INFO: Container coredns ready: true, restart count 1 Aug 27 23:50:47.771: INFO: node-exporter-z2ngr started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded) Aug 27 23:50:47.771: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 27 23:50:47.771: INFO: Container node-exporter ready: true, restart count 0 Aug 27 23:50:47.771: INFO: kube-scheduler-master1 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.771: INFO: Container kube-scheduler ready: true, restart count 0 Aug 27 23:50:47.771: INFO: kube-flannel-pp7vp started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded) Aug 27 23:50:47.771: INFO: Init container install-cni ready: true, restart count 0 Aug 27 23:50:47.771: INFO: Container kube-flannel ready: true, restart count 3 Aug 27 23:50:47.771: INFO: kube-multus-ds-amd64-sfr9k started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.771: INFO: Container kube-multus ready: true, restart count 1 Aug 27 23:50:47.771: INFO: docker-registry-docker-registry-56cbc7bc58-cthtt started at 2021-08-27 20:51:47 +0000 UTC (0+2 container statuses recorded) Aug 27 23:50:47.771: INFO: Container docker-registry ready: true, restart count 0 Aug 27 23:50:47.771: INFO: Container nginx ready: true, restart count 0 W0827 23:50:47.782709 26 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. Aug 27 23:50:47.806: INFO: Latency metrics for node master1 Aug 27 23:50:47.806: INFO: Logging node info for node master2 Aug 27 23:50:47.808: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 666e473f-d9e6-4c06-8b56-06474d788f70 61102 0 2021-08-27 20:46:51 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"42:86:ff:30:bd:4d"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-08-27 20:46:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {kubelet Update v1 2021-08-27 20:46:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-08-27 20:48:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-08-27 20:55:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:51:53 +0000 UTC,LastTransitionTime:2021-08-27 20:51:53 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-27 23:50:38 +0000 UTC,LastTransitionTime:2021-08-27 20:46:51 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-27 23:50:38 +0000 UTC,LastTransitionTime:2021-08-27 20:46:51 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-27 23:50:38 +0000 UTC,LastTransitionTime:2021-08-27 20:46:51 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-27 23:50:38 +0000 UTC,LastTransitionTime:2021-08-27 20:48:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:2835065974a64998811b9acd85de209b,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:309b9155-1e2a-4ebd-900f-bba5abfc3a5d,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[sirot/netperf-latest@sha256:23929b922bb077cb341450fc1c90ae42e6f75da5d7ce61cd1880b08560a5ff85 sirot/netperf-latest:latest],SizeBytes:282025213,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Aug 27 23:50:47.808: INFO: Logging kubelet events for node master2 Aug 27 23:50:47.811: INFO: Logging pods the kubelet thinks is on node master2 Aug 27 23:50:47.823: INFO: kube-controller-manager-master2 started at 2021-08-27 20:51:01 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.823: INFO: Container kube-controller-manager ready: true, restart count 2 Aug 27 23:50:47.823: INFO: kube-flannel-4znnq started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded) Aug 27 23:50:47.823: INFO: Init container install-cni ready: true, restart count 2 Aug 27 23:50:47.823: INFO: Container kube-flannel ready: true, restart count 2 Aug 27 23:50:47.823: INFO: kube-multus-ds-amd64-4mgbk started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.823: INFO: Container kube-multus ready: true, restart count 1 Aug 27 23:50:47.823: INFO: prometheus-operator-5bb8cb9d8f-whr5p started at 2021-08-27 20:59:06 +0000 UTC (0+2 container statuses recorded) Aug 27 23:50:47.823: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 27 23:50:47.823: INFO: Container prometheus-operator ready: true, restart count 0 Aug 27 23:50:47.823: INFO: node-exporter-96jk5 started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded) Aug 27 23:50:47.823: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 27 23:50:47.823: INFO: Container node-exporter ready: true, restart count 0 Aug 27 23:50:47.823: INFO: kube-apiserver-master2 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.823: INFO: Container kube-apiserver ready: true, restart count 0 Aug 27 23:50:47.823: INFO: kube-scheduler-master2 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.823: INFO: Container kube-scheduler ready: true, restart count 2 Aug 27 23:50:47.823: INFO: kube-proxy-b4mn9 started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.823: INFO: Container kube-proxy ready: true, restart count 1 Aug 27 23:50:47.823: INFO: node-feature-discovery-controller-5bf5c49849-zr9zd started at 2021-08-27 20:55:09 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.823: INFO: Container nfd-controller ready: true, restart count 0 W0827 23:50:47.834394 26 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. Aug 27 23:50:47.859: INFO: Latency metrics for node master2 Aug 27 23:50:47.859: INFO: Logging node info for node master3 Aug 27 23:50:47.862: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 620b6dba-f2c5-46e9-b2ff-d2f4197167d0 61225 0 2021-08-27 20:47:02 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ce:de:ea:c3:40:4f"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-08-27 20:47:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-08-27 20:47:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-08-27 20:48:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:51:44 +0000 UTC,LastTransitionTime:2021-08-27 20:51:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-27 23:50:46 +0000 UTC,LastTransitionTime:2021-08-27 20:47:02 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-27 23:50:46 +0000 UTC,LastTransitionTime:2021-08-27 20:47:02 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-27 23:50:46 +0000 UTC,LastTransitionTime:2021-08-27 20:47:02 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-27 23:50:46 +0000 UTC,LastTransitionTime:2021-08-27 20:48:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:37decbffe0e84048b5801289ad3be5bf,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:c96fe8b9-1ce0-44cd-935a-b58987e26570,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496891,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Aug 27 23:50:47.862: INFO: Logging kubelet events for node master3 Aug 27 23:50:47.864: INFO: Logging pods the kubelet thinks is on node master3 Aug 27 23:50:47.874: INFO: kube-controller-manager-master3 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.874: INFO: Container kube-controller-manager ready: true, restart count 2 Aug 27 23:50:47.874: INFO: kube-proxy-8sxhm started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.874: INFO: Container kube-proxy ready: true, restart count 1 Aug 27 23:50:47.874: INFO: kube-flannel-fkz5d started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded) Aug 27 23:50:47.874: INFO: Init container install-cni ready: true, restart count 0 Aug 27 23:50:47.874: INFO: Container kube-flannel ready: true, restart count 1 Aug 27 23:50:47.874: INFO: dns-autoscaler-5b7b5c9b6f-54xch started at 2021-08-27 20:49:19 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.874: INFO: Container autoscaler ready: true, restart count 1 Aug 27 23:50:47.874: INFO: coredns-7677f9bb54-rxplt started at 2021-08-27 20:49:21 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.874: INFO: Container coredns ready: true, restart count 1 Aug 27 23:50:47.874: INFO: kube-apiserver-master3 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.874: INFO: Container kube-apiserver ready: true, restart count 0 Aug 27 23:50:47.874: INFO: kube-scheduler-master3 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.874: INFO: Container kube-scheduler ready: true, restart count 2 Aug 27 23:50:47.874: INFO: kube-multus-ds-amd64-wwcgv started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.874: INFO: Container kube-multus ready: true, restart count 1 Aug 27 23:50:47.874: INFO: node-exporter-d4m7q started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded) Aug 27 23:50:47.874: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 27 23:50:47.874: INFO: Container node-exporter ready: true, restart count 0 W0827 23:50:47.887754 26 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. Aug 27 23:50:47.910: INFO: Latency metrics for node master3 Aug 27 23:50:47.910: INFO: Logging node info for node node1 Aug 27 23:50:47.913: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 e7a2481e-32db-4c83-bd9f-4a0687258e7a 61131 0 2021-08-27 20:48:09 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.36.2.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"56:c7:37:40:51:ca"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-08-27 20:48:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubeadm Update v1 2021-08-27 20:48:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-08-27 20:55:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-08-27 20:57:40 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-08-27 20:57:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269628928 0} {<nil>} 196552372Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884628480 0} {<nil>} 174692020Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:51:49 +0000 UTC,LastTransitionTime:2021-08-27 20:51:49 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-27 23:50:40 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-27 23:50:40 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-27 23:50:40 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-27 23:50:40 +0000 UTC,LastTransitionTime:2021-08-27 20:48:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:c1e38e80ea114a5f96601202301ce842,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:e769e86d-15c0-442c-a93b-bcc6c33ff1cd,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:331c6faa8b0d5ec72cf105e87d35df0a2f2baeec3d6217a51faa73f9460f937f localhost:30500/barometer-collectd:stable],SizeBytes:1238704157,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:a7cea43d9d2f67c38fbf0407786edbe660ee9072945f7bb272b55fd255e8eaca opnfv/barometer-collectd:stable],SizeBytes:1075746799,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:1003787960,},ContainerImage{Names:[localhost:30500/cmk@sha256:fd1487b0c07556a087eff669e70c501a704720dcd53ff75183593de6720585f2 cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[aquasec/kube-hunter@sha256:2be6820bc1d7e0f57193a9a27d5a3e16b2fd93c53747b03ce8ca48c6fc323781 aquasec/kube-hunter:0.3.1],SizeBytes:347611549,},ContainerImage{Names:[golang@sha256:db2475a1dbb2149508e5db31d7d77a75e6600d54be645f37681f03f2762169ba golang:alpine3.12],SizeBytes:301186719,},ContainerImage{Names:[sirot/netperf-latest@sha256:23929b922bb077cb341450fc1c90ae42e6f75da5d7ce61cd1880b08560a5ff85 sirot/netperf-latest:latest],SizeBytes:282025213,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:d7300ccf7ff3e9cea2111d275143b8050618bbc1d1ffe41f46286b1696261243 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44393508,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:de25c7fc6c4f3a27c7f0c2dff454e4671823a34d88abd533f210848d527e0fbb alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Aug 27 23:50:47.914: INFO: Logging kubelet events for node node1 Aug 27 23:50:47.916: INFO: Logging pods the kubelet thinks is on node node1 Aug 27 23:50:47.931: INFO: nginx-proxy-node1 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.931: INFO: Container nginx-proxy ready: true, restart count 2 Aug 27 23:50:47.931: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-9lndx started at 2021-08-27 20:55:51 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.931: INFO: Container kube-sriovdp ready: true, restart count 0 Aug 27 23:50:47.931: INFO: pod-init-0f2a9f46-6fd7-4394-8340-4707adea2272 started at 2021-08-27 23:45:54 +0000 UTC (2+1 container statuses recorded) Aug 27 23:50:47.931: INFO: Init container init1 ready: false, restart count 0 Aug 27 23:50:47.931: INFO: Init container init2 ready: false, restart count 0 Aug 27 23:50:47.931: INFO: Container run1 ready: false, restart count 0 Aug 27 23:50:47.931: INFO: pod-init-005a6e95-bd11-430c-ad1a-d87bdbc991f8 started at 2021-08-27 23:46:46 +0000 UTC (2+1 container statuses recorded) Aug 27 23:50:47.931: INFO: Init container init1 ready: false, restart count 0 Aug 27 23:50:47.931: INFO: Init container init2 ready: false, restart count 0 Aug 27 23:50:47.931: INFO: Container run1 ready: false, restart count 0 Aug 27 23:50:47.931: INFO: node-exporter-4cvlq started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded) Aug 27 23:50:47.931: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 27 23:50:47.931: INFO: Container node-exporter ready: true, restart count 0 Aug 27 23:50:47.931: INFO: kubernetes-dashboard-86c6f9df5b-c56fg started at 2021-08-27 20:49:21 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.931: INFO: Container kubernetes-dashboard ready: true, restart count 1 Aug 27 23:50:47.931: INFO: node-feature-discovery-worker-bd9kg started at 2021-08-27 20:55:06 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.932: INFO: Container nfd-worker ready: true, restart count 0 Aug 27 23:50:47.932: INFO: ss2-0 started at 2021-08-27 23:46:41 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.932: INFO: Container webserver ready: false, restart count 0 Aug 27 23:50:47.932: INFO: kube-proxy-pb5bl started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.932: INFO: Container kube-proxy ready: true, restart count 2 Aug 27 23:50:47.932: INFO: collectd-ccvwg started at 2021-08-27 21:04:15 +0000 UTC (0+3 container statuses recorded) Aug 27 23:50:47.932: INFO: Container collectd ready: true, restart count 0 Aug 27 23:50:47.932: INFO: Container collectd-exporter ready: true, restart count 0 Aug 27 23:50:47.932: INFO: Container rbac-proxy ready: true, restart count 0 Aug 27 23:50:47.932: INFO: prometheus-k8s-0 started at 2021-08-27 20:59:29 +0000 UTC (0+5 container statuses recorded) Aug 27 23:50:47.932: INFO: Container custom-metrics-apiserver ready: true, restart count 0 Aug 27 23:50:47.932: INFO: Container grafana ready: true, restart count 0 Aug 27 23:50:47.932: INFO: Container prometheus ready: true, restart count 1 Aug 27 23:50:47.932: INFO: Container prometheus-config-reloader ready: true, restart count 0 Aug 27 23:50:47.932: INFO: Container rules-configmap-reloader ready: true, restart count 0 Aug 27 23:50:47.932: INFO: kube-flannel-ssxn7 started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded) Aug 27 23:50:47.932: INFO: Init container install-cni ready: true, restart count 0 Aug 27 23:50:47.932: INFO: Container kube-flannel ready: true, restart count 1 Aug 27 23:50:47.932: INFO: kube-multus-ds-amd64-nn7bl started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.932: INFO: Container kube-multus ready: true, restart count 1 Aug 27 23:50:47.932: INFO: kubernetes-metrics-scraper-678c97765c-gtp5x started at 2021-08-27 20:49:21 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.932: INFO: Container kubernetes-metrics-scraper ready: true, restart count 1 Aug 27 23:50:47.932: INFO: cmk-init-discover-node1-spg26 started at 2021-08-27 20:57:37 +0000 UTC (0+3 container statuses recorded) Aug 27 23:50:47.932: INFO: Container discover ready: false, restart count 0 Aug 27 23:50:47.932: INFO: Container init ready: false, restart count 0 Aug 27 23:50:47.932: INFO: Container install ready: false, restart count 0 Aug 27 23:50:47.932: INFO: cmk-jw4m6 started at 2021-08-27 20:58:19 +0000 UTC (0+2 container statuses recorded) Aug 27 23:50:47.932: INFO: Container nodereport ready: true, restart count 0 Aug 27 23:50:47.932: INFO: Container reconcile ready: true, restart count 0 W0827 23:50:47.945406 26 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. Aug 27 23:50:47.977: INFO: Latency metrics for node node1 Aug 27 23:50:47.977: INFO: Logging node info for node node2 Aug 27 23:50:47.979: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 eb7da7c0-513f-4072-a078-ad3d24f88114 61108 0 2021-08-27 20:48:09 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.36.2.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"9a:88:9c:d1:39:68"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-08-27 20:48:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubeadm Update v1 2021-08-27 20:48:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-08-27 20:55:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-08-27 20:58:03 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-08-27 20:58:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269628928 0} {<nil>} 196552372Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884628480 0} {<nil>} 174692020Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:50:36 +0000 UTC,LastTransitionTime:2021-08-27 20:50:36 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-27 23:50:38 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-27 23:50:38 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-27 23:50:38 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-27 23:50:38 +0000 UTC,LastTransitionTime:2021-08-27 20:48:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:0ccfc2a4a9b7400c9ca53b5de0ca4970,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:26b02690-0814-4c92-9f6d-d315df796ce6,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:331c6faa8b0d5ec72cf105e87d35df0a2f2baeec3d6217a51faa73f9460f937f localhost:30500/barometer-collectd:stable],SizeBytes:1238704157,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[localhost:30500/cmk@sha256:fd1487b0c07556a087eff669e70c501a704720dcd53ff75183593de6720585f2 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:d7300ccf7ff3e9cea2111d275143b8050618bbc1d1ffe41f46286b1696261243 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44393508,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:ae72171f047a37ee5423e0692df7429830919af16e9d668ab0c80b723863d102 localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:cd98d1edca8e5e2e3ea42cbc463812483e5d069d10f0974ca9d484b5a7bd68db localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[aquasec/kube-bench@sha256:3544f6662feb73d36fdba35b17652e2fd73aae45bd4b60e76d7ab928220b3cc6 aquasec/kube-bench:0.3.1],SizeBytes:19301876,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Aug 27 23:50:47.980: INFO: Logging kubelet events for node node2 Aug 27 23:50:47.982: INFO: Logging pods the kubelet thinks is on node node2 Aug 27 23:50:47.993: INFO: bin-false1947139b-d6b9-49d8-b4c7-422c259d1e78 started at 2021-08-27 23:49:00 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.993: INFO: Container bin-false1947139b-d6b9-49d8-b4c7-422c259d1e78 ready: false, restart count 0 Aug 27 23:50:47.993: INFO: collectd-64dp2 started at 2021-08-27 21:04:15 +0000 UTC (0+3 container statuses recorded) Aug 27 23:50:47.993: INFO: Container collectd ready: true, restart count 0 Aug 27 23:50:47.993: INFO: Container collectd-exporter ready: true, restart count 0 Aug 27 23:50:47.993: INFO: Container rbac-proxy ready: true, restart count 0 Aug 27 23:50:47.993: INFO: ss-0 started at 2021-08-27 23:45:13 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.993: INFO: Container webserver ready: false, restart count 0 Aug 27 23:50:47.993: INFO: cmk-webhook-6c9d5f8578-ndbx2 started at 2021-08-27 20:58:20 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.993: INFO: Container cmk-webhook ready: true, restart count 0 Aug 27 23:50:47.993: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-v99df started at 2021-08-27 21:02:08 +0000 UTC (0+2 container statuses recorded) Aug 27 23:50:47.993: INFO: Container tas-controller ready: true, restart count 0 Aug 27 23:50:47.993: INFO: Container tas-extender ready: true, restart count 0 Aug 27 23:50:47.993: INFO: kube-flannel-t9qv4 started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded) Aug 27 23:50:47.993: INFO: Init container install-cni ready: true, restart count 0 Aug 27 23:50:47.993: INFO: Container kube-flannel ready: true, restart count 2 Aug 27 23:50:47.993: INFO: cmk-init-discover-node2-l9qjd started at 2021-08-27 20:57:57 +0000 UTC (0+3 container statuses recorded) Aug 27 23:50:47.993: INFO: Container discover ready: false, restart count 0 Aug 27 23:50:47.993: INFO: Container init ready: false, restart count 0 Aug 27 23:50:47.993: INFO: Container install ready: false, restart count 0 Aug 27 23:50:47.993: INFO: node-feature-discovery-worker-54lfh started at 2021-08-27 20:55:06 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.993: INFO: Container nfd-worker ready: true, restart count 0 Aug 27 23:50:47.993: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4f962 started at 2021-08-27 20:55:51 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.993: INFO: Container kube-sriovdp ready: true, restart count 0 Aug 27 23:50:47.993: INFO: termination-message-container3c8c3ba7-b0af-4dff-bf19-3f698c671341 started at 2021-08-27 23:50:23 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.993: INFO: Container termination-message-container ready: false, restart count 0 Aug 27 23:50:47.993: INFO: pod-update-eaf44ddd-7329-4952-b30f-2025d9bf8fc2 started at 2021-08-27 23:47:37 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.993: INFO: Container nginx ready: false, restart count 0 Aug 27 23:50:47.993: INFO: nginx-proxy-node2 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.993: INFO: Container nginx-proxy ready: true, restart count 1 Aug 27 23:50:47.993: INFO: kube-proxy-r4q4t started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.993: INFO: Container kube-proxy ready: true, restart count 2 Aug 27 23:50:47.993: INFO: kube-multus-ds-amd64-tfffk started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.993: INFO: Container kube-multus ready: true, restart count 1 Aug 27 23:50:47.993: INFO: node-exporter-p6h5h started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded) Aug 27 23:50:47.993: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 27 23:50:47.993: INFO: Container node-exporter ready: true, restart count 0 Aug 27 23:50:47.993: INFO: ss-0 started at 2021-08-27 23:47:20 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.993: INFO: Container webserver ready: false, restart count 0 Aug 27 23:50:47.993: INFO: busybox-readonly-false-f7f9026d-2e01-458b-928e-8960287201ef started at 2021-08-27 23:50:47 +0000 UTC (0+1 container statuses recorded) Aug 27 23:50:47.993: INFO: Container busybox-readonly-false-f7f9026d-2e01-458b-928e-8960287201ef ready: false, restart count 0 Aug 27 23:50:47.993: INFO: cmk-fzjgr started at 2021-08-27 20:58:20 +0000 UTC (0+2 container statuses recorded) Aug 27 23:50:47.993: INFO: Container nodereport ready: true, restart count 0 Aug 27 23:50:47.993: INFO: Container reconcile ready: true, restart count 0 W0827 23:50:48.006388 26 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. Aug 27 23:50:48.079: INFO: Latency metrics for node node2 Aug 27 23:50:48.079: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "container-probe-6247" for this suite. /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 Aug 27 23:55:52.751: Unexpected error: <*errors.errorString | 0xc00208d3f0>: { s: "expected pod \"var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a\" success: Gave up after waiting 5m0s for pod \"var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a\" to be \"Succeeded or Failed\"", } expected pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a" success: Gave up after waiting 5m0s for pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a" to be "Succeeded or Failed" occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:725 [BeforeEach] [k8s.io] Variable Expansion /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client Aug 27 23:50:48.124: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename var-expansion STEP: Waiting for a default service account to be provisioned in namespace [It] should allow substituting values in a container's args [NodeConformance] [Conformance] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 STEP: Creating a pod to test substitution in container's args Aug 27 23:50:48.159: INFO: Waiting up to 5m0s for pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a" in namespace "var-expansion-6695" to be "Succeeded or Failed" Aug 27 23:50:48.163: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 3.871247ms Aug 27 23:50:50.166: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 2.006597412s Aug 27 23:50:52.170: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 4.010830138s Aug 27 23:50:54.173: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 6.013739161s Aug 27 23:50:56.176: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 8.016908166s Aug 27 23:50:58.181: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 10.021170391s Aug 27 23:51:00.184: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 12.024644627s Aug 27 23:51:02.188: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 14.028286183s Aug 27 23:51:04.191: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 16.032092224s Aug 27 23:51:06.195: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 18.035991558s Aug 27 23:51:08.198: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 20.039112633s Aug 27 23:51:10.202: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 22.042745532s Aug 27 23:51:12.206: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 24.046861391s Aug 27 23:51:14.209: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 26.050132682s Aug 27 23:51:16.212: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 28.052811114s Aug 27 23:51:18.216: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 30.05626826s Aug 27 23:51:20.219: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 32.059270965s Aug 27 23:51:22.222: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 34.06257195s Aug 27 23:51:24.226: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 36.066879992s Aug 27 23:51:26.230: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 38.070300574s Aug 27 23:51:28.236: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 40.076607744s Aug 27 23:51:30.239: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 42.079649658s Aug 27 23:51:32.242: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 44.082903459s Aug 27 23:51:34.246: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 46.087115895s Aug 27 23:51:36.251: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 48.091469941s Aug 27 23:51:38.255: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 50.095612748s Aug 27 23:51:40.261: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 52.101541617s Aug 27 23:51:42.265: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 54.105167224s Aug 27 23:51:44.268: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 56.108944873s Aug 27 23:51:46.274: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 58.114192769s Aug 27 23:51:48.276: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.117009055s Aug 27 23:51:50.282: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.122659727s Aug 27 23:51:52.285: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.126099107s Aug 27 23:51:54.290: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.130184091s Aug 27 23:51:56.293: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.133720007s Aug 27 23:51:58.296: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.13673138s Aug 27 23:52:00.299: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.139947764s Aug 27 23:52:02.303: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.143259931s Aug 27 23:52:04.308: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.148160892s Aug 27 23:52:06.312: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.152977654s Aug 27 23:52:08.317: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.158094299s Aug 27 23:52:10.321: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.161306854s Aug 27 23:52:12.325: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.165517694s Aug 27 23:52:14.330: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.170441166s Aug 27 23:52:16.333: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.173998536s Aug 27 23:52:18.337: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.178060094s Aug 27 23:52:20.341: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.181807294s Aug 27 23:52:22.346: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.186844743s Aug 27 23:52:24.352: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.192260707s Aug 27 23:52:26.355: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.196032754s Aug 27 23:52:28.360: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.200667654s Aug 27 23:52:30.364: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.204250493s Aug 27 23:52:32.366: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.206793627s Aug 27 23:52:34.372: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.2123608s Aug 27 23:52:36.375: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.216051036s Aug 27 23:52:38.381: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.22159162s Aug 27 23:52:40.384: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.224266818s Aug 27 23:52:42.388: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.228363696s Aug 27 23:52:44.392: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.233027726s Aug 27 23:52:46.397: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.237371694s Aug 27 23:52:48.400: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.241027693s Aug 27 23:52:50.403: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.243879161s Aug 27 23:52:52.406: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.246898454s Aug 27 23:52:54.409: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.249751928s Aug 27 23:52:56.413: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.253341561s Aug 27 23:52:58.416: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.257141317s Aug 27 23:53:00.420: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.260240101s Aug 27 23:53:02.424: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.264390751s Aug 27 23:53:04.428: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.268994627s Aug 27 23:53:06.434: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.274733405s Aug 27 23:53:08.439: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.279393757s Aug 27 23:53:10.442: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.282755223s Aug 27 23:53:12.447: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.287483321s Aug 27 23:53:14.452: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.292447348s Aug 27 23:53:16.457: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.297454728s Aug 27 23:53:18.462: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.302875496s Aug 27 23:53:20.465: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.306031631s Aug 27 23:53:22.470: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.310955923s Aug 27 23:53:24.473: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.313965752s Aug 27 23:53:26.477: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.317440647s Aug 27 23:53:28.481: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.321186166s Aug 27 23:53:30.484: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.32441944s Aug 27 23:53:32.487: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.327644315s Aug 27 23:53:34.491: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.332125774s Aug 27 23:53:36.496: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.336218361s Aug 27 23:53:38.501: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.34134162s Aug 27 23:53:40.504: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.344837116s Aug 27 23:53:42.509: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.349575894s Aug 27 23:53:44.513: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.353376043s Aug 27 23:53:46.517: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.35717939s Aug 27 23:53:48.520: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.360361437s Aug 27 23:53:50.523: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.364010695s Aug 27 23:53:52.527: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.367547751s Aug 27 23:53:54.531: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.372022063s Aug 27 23:53:56.535: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.37557803s Aug 27 23:53:58.539: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.379444802s Aug 27 23:54:00.544: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.384519662s Aug 27 23:54:02.548: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.388145897s Aug 27 23:54:04.551: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.391480991s Aug 27 23:54:06.555: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.395348532s Aug 27 23:54:08.559: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.399501849s Aug 27 23:54:10.563: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.403452569s Aug 27 23:54:12.568: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.408239087s Aug 27 23:54:14.571: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.412056403s Aug 27 23:54:16.575: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.415713124s Aug 27 23:54:18.579: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.420071409s Aug 27 23:54:20.582: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.423108792s Aug 27 23:54:22.586: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.427112657s Aug 27 23:54:24.591: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.43178755s Aug 27 23:54:26.594: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.434838033s Aug 27 23:54:28.597: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.438088983s Aug 27 23:54:30.601: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.441507626s Aug 27 23:54:32.605: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.445149194s Aug 27 23:54:34.608: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.448631473s Aug 27 23:54:36.611: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.451424168s Aug 27 23:54:38.615: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.455883624s Aug 27 23:54:40.618: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.459082605s Aug 27 23:54:42.622: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.462228708s Aug 27 23:54:44.624: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.465064801s Aug 27 23:54:46.627: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.467774589s Aug 27 23:54:48.631: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.471231501s Aug 27 23:54:50.635: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.47528949s Aug 27 23:54:52.638: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.478718869s Aug 27 23:54:54.642: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.482238878s Aug 27 23:54:56.644: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.485105428s Aug 27 23:54:58.648: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.488198111s Aug 27 23:55:00.651: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.491373075s Aug 27 23:55:02.653: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.494066042s Aug 27 23:55:04.659: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.499392517s Aug 27 23:55:06.662: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.502200699s Aug 27 23:55:08.666: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.506162936s Aug 27 23:55:10.670: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.510188364s Aug 27 23:55:12.672: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.512958101s Aug 27 23:55:14.677: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.517751359s Aug 27 23:55:16.682: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.522154082s Aug 27 23:55:18.687: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.527271727s Aug 27 23:55:20.690: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.530744373s Aug 27 23:55:22.693: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.533796408s Aug 27 23:55:24.696: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.536318382s Aug 27 23:55:26.699: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.53925818s Aug 27 23:55:28.701: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.542001591s Aug 27 23:55:30.704: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.544730658s Aug 27 23:55:32.707: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.547958582s Aug 27 23:55:34.711: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.551436887s Aug 27 23:55:36.714: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.555056606s Aug 27 23:55:38.718: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.559138059s Aug 27 23:55:40.722: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.562934704s Aug 27 23:55:42.728: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.568428883s Aug 27 23:55:44.731: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.571369744s Aug 27 23:55:46.734: INFO: Pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.574407676s Aug 27 23:55:48.740: INFO: Failed to get logs from node "node2" pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a" container "dapi-container": the server rejected our request for an unknown reason (get pods var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a) STEP: delete the pod Aug 27 23:55:48.745: INFO: Waiting for pod var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a to disappear Aug 27 23:55:48.747: INFO: Pod var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a still exists Aug 27 23:55:50.748: INFO: Waiting for pod var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a to disappear Aug 27 23:55:50.750: INFO: Pod var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a still exists Aug 27 23:55:52.748: INFO: Waiting for pod var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a to disappear Aug 27 23:55:52.751: INFO: Pod var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a no longer exists Aug 27 23:55:52.751: FAIL: Unexpected error: <*errors.errorString | 0xc00208d3f0>: { s: "expected pod \"var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a\" success: Gave up after waiting 5m0s for pod \"var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a\" to be \"Succeeded or Failed\"", } expected pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a" success: Gave up after waiting 5m0s for pod "var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a" to be "Succeeded or Failed" occurred Full Stack Trace k8s.io/kubernetes/test/e2e/framework.(*Framework).testContainerOutputMatcher(0xc001170160, 0x4c7aa14, 0x20, 0xc002abc400, 0x0, 0xc0012ed1c8, 0x1, 0x1, 0x4df0110) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:725 +0x1ee k8s.io/kubernetes/test/e2e/framework.(*Framework).TestContainerOutput(...) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:525 k8s.io/kubernetes/test/e2e/common.glob..func9.3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/expansion.go:99 +0x1fb k8s.io/kubernetes/test/e2e.RunE2ETests(0xc00180a780) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc00180a780) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc00180a780, 0x4dec428) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [k8s.io] Variable Expansion /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "var-expansion-6695". STEP: Found 9 events. Aug 27 23:55:52.757: INFO: At 2021-08-27 23:50:48 +0000 UTC - event for var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a: {default-scheduler } Scheduled: Successfully assigned var-expansion-6695/var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a to node2 Aug 27 23:55:52.757: INFO: At 2021-08-27 23:50:50 +0000 UTC - event for var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a: {multus } AddedInterface: Add eth0 [10.244.4.64/24] Aug 27 23:55:52.757: INFO: At 2021-08-27 23:50:50 +0000 UTC - event for var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a: {kubelet node2} Pulling: Pulling image "docker.io/library/busybox:1.29" Aug 27 23:55:52.757: INFO: At 2021-08-27 23:50:51 +0000 UTC - event for var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a: {kubelet node2} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit Aug 27 23:55:52.757: INFO: At 2021-08-27 23:50:51 +0000 UTC - event for var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a: {kubelet node2} Failed: Error: ErrImagePull Aug 27 23:55:52.757: INFO: At 2021-08-27 23:50:52 +0000 UTC - event for var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a: {kubelet node2} SandboxChanged: Pod sandbox changed, it will be killed and re-created. Aug 27 23:55:52.757: INFO: At 2021-08-27 23:50:53 +0000 UTC - event for var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a: {multus } AddedInterface: Add eth0 [10.244.4.65/24] Aug 27 23:55:52.757: INFO: At 2021-08-27 23:50:53 +0000 UTC - event for var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a: {kubelet node2} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" Aug 27 23:55:52.757: INFO: At 2021-08-27 23:50:53 +0000 UTC - event for var-expansion-96d10a0a-98a2-412f-90ad-9d596105624a: {kubelet node2} Failed: Error: ImagePullBackOff Aug 27 23:55:52.759: INFO: POD NODE PHASE GRACE CONDITIONS Aug 27 23:55:52.759: INFO: Aug 27 23:55:52.763: INFO: Logging node info for node master1 Aug 27 23:55:52.766: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 3af53387-5aee-42c1-b0e7-644cf9161d48 63215 0 2021-08-27 20:46:13 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"92:f8:b6:72:e4:be"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-08-27 20:46:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-08-27 20:46:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {kube-controller-manager Update v1 2021-08-27 20:46:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:50:56 +0000 UTC,LastTransitionTime:2021-08-27 20:50:56 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-27 23:55:51 +0000 UTC,LastTransitionTime:2021-08-27 20:46:13 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-27 23:55:51 +0000 UTC,LastTransitionTime:2021-08-27 20:46:13 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-27 23:55:51 +0000 UTC,LastTransitionTime:2021-08-27 20:46:13 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-27 23:55:51 +0000 UTC,LastTransitionTime:2021-08-27 20:50:30 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:5d584023135a46ecb77596bf48ed7f2f,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:587cafa0-6de3-49f8-906e-06315a8ff104,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[sirot/netperf-latest@sha256:23929b922bb077cb341450fc1c90ae42e6f75da5d7ce61cd1880b08560a5ff85 sirot/netperf-latest:latest],SizeBytes:282025213,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:ae72171f047a37ee5423e0692df7429830919af16e9d668ab0c80b723863d102 tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:cd98d1edca8e5e2e3ea42cbc463812483e5d069d10f0974ca9d484b5a7bd68db tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[aquasec/kube-bench@sha256:3544f6662feb73d36fdba35b17652e2fd73aae45bd4b60e76d7ab928220b3cc6 aquasec/kube-bench:0.3.1],SizeBytes:19301876,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Aug 27 23:55:52.766: INFO: Logging kubelet events for node master1 Aug 27 23:55:52.768: INFO: Logging pods the kubelet thinks is on node master1 Aug 27 23:55:52.777: INFO: kube-scheduler-master1 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:52.777: INFO: Container kube-scheduler ready: true, restart count 0 Aug 27 23:55:52.777: INFO: kube-flannel-pp7vp started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded) Aug 27 23:55:52.777: INFO: Init container install-cni ready: true, restart count 0 Aug 27 23:55:52.777: INFO: Container kube-flannel ready: true, restart count 3 Aug 27 23:55:52.777: INFO: kube-multus-ds-amd64-sfr9k started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:52.777: INFO: Container kube-multus ready: true, restart count 1 Aug 27 23:55:52.777: INFO: docker-registry-docker-registry-56cbc7bc58-cthtt started at 2021-08-27 20:51:47 +0000 UTC (0+2 container statuses recorded) Aug 27 23:55:52.777: INFO: Container docker-registry ready: true, restart count 0 Aug 27 23:55:52.777: INFO: Container nginx ready: true, restart count 0 Aug 27 23:55:52.777: INFO: kube-apiserver-master1 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:52.777: INFO: Container kube-apiserver ready: true, restart count 0 Aug 27 23:55:52.777: INFO: kube-controller-manager-master1 started at 2021-08-27 20:54:27 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:52.777: INFO: Container kube-controller-manager ready: true, restart count 3 Aug 27 23:55:52.777: INFO: kube-proxy-rb5p6 started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:52.777: INFO: Container kube-proxy ready: true, restart count 1 Aug 27 23:55:52.777: INFO: coredns-7677f9bb54-dwtp5 started at 2021-08-27 20:49:16 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:52.777: INFO: Container coredns ready: true, restart count 1 Aug 27 23:55:52.777: INFO: node-exporter-z2ngr started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded) Aug 27 23:55:52.777: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 27 23:55:52.777: INFO: Container node-exporter ready: true, restart count 0 W0827 23:55:52.788985 26 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. Aug 27 23:55:52.818: INFO: Latency metrics for node master1 Aug 27 23:55:52.818: INFO: Logging node info for node master2 Aug 27 23:55:52.821: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 666e473f-d9e6-4c06-8b56-06474d788f70 63182 0 2021-08-27 20:46:51 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"42:86:ff:30:bd:4d"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-08-27 20:46:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {kubelet Update v1 2021-08-27 20:46:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-08-27 20:48:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-08-27 20:55:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:51:53 +0000 UTC,LastTransitionTime:2021-08-27 20:51:53 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-27 23:55:49 +0000 UTC,LastTransitionTime:2021-08-27 20:46:51 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-27 23:55:49 +0000 UTC,LastTransitionTime:2021-08-27 20:46:51 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-27 23:55:49 +0000 UTC,LastTransitionTime:2021-08-27 20:46:51 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-27 23:55:49 +0000 UTC,LastTransitionTime:2021-08-27 20:48:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:2835065974a64998811b9acd85de209b,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:309b9155-1e2a-4ebd-900f-bba5abfc3a5d,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[sirot/netperf-latest@sha256:23929b922bb077cb341450fc1c90ae42e6f75da5d7ce61cd1880b08560a5ff85 sirot/netperf-latest:latest],SizeBytes:282025213,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Aug 27 23:55:52.822: INFO: Logging kubelet events for node master2 Aug 27 23:55:52.824: INFO: Logging pods the kubelet thinks is on node master2 Aug 27 23:55:52.832: INFO: node-feature-discovery-controller-5bf5c49849-zr9zd started at 2021-08-27 20:55:09 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:52.832: INFO: Container nfd-controller ready: true, restart count 0 Aug 27 23:55:52.832: INFO: kube-apiserver-master2 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:52.832: INFO: Container kube-apiserver ready: true, restart count 0 Aug 27 23:55:52.832: INFO: kube-scheduler-master2 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:52.832: INFO: Container kube-scheduler ready: true, restart count 2 Aug 27 23:55:52.832: INFO: kube-proxy-b4mn9 started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:52.832: INFO: Container kube-proxy ready: true, restart count 1 Aug 27 23:55:52.832: INFO: prometheus-operator-5bb8cb9d8f-whr5p started at 2021-08-27 20:59:06 +0000 UTC (0+2 container statuses recorded) Aug 27 23:55:52.832: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 27 23:55:52.832: INFO: Container prometheus-operator ready: true, restart count 0 Aug 27 23:55:52.832: INFO: node-exporter-96jk5 started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded) Aug 27 23:55:52.832: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 27 23:55:52.832: INFO: Container node-exporter ready: true, restart count 0 Aug 27 23:55:52.832: INFO: kube-controller-manager-master2 started at 2021-08-27 20:51:01 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:52.832: INFO: Container kube-controller-manager ready: true, restart count 2 Aug 27 23:55:52.832: INFO: kube-flannel-4znnq started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded) Aug 27 23:55:52.832: INFO: Init container install-cni ready: true, restart count 2 Aug 27 23:55:52.832: INFO: Container kube-flannel ready: true, restart count 2 Aug 27 23:55:52.832: INFO: kube-multus-ds-amd64-4mgbk started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:52.832: INFO: Container kube-multus ready: true, restart count 1 W0827 23:55:52.847062 26 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. Aug 27 23:55:52.877: INFO: Latency metrics for node master2 Aug 27 23:55:52.877: INFO: Logging node info for node master3 Aug 27 23:55:52.880: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 620b6dba-f2c5-46e9-b2ff-d2f4197167d0 63167 0 2021-08-27 20:47:02 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ce:de:ea:c3:40:4f"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-08-27 20:47:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-08-27 20:47:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-08-27 20:48:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:51:44 +0000 UTC,LastTransitionTime:2021-08-27 20:51:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-27 23:55:48 +0000 UTC,LastTransitionTime:2021-08-27 20:47:02 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-27 23:55:48 +0000 UTC,LastTransitionTime:2021-08-27 20:47:02 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-27 23:55:48 +0000 UTC,LastTransitionTime:2021-08-27 20:47:02 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-27 23:55:48 +0000 UTC,LastTransitionTime:2021-08-27 20:48:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:37decbffe0e84048b5801289ad3be5bf,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:c96fe8b9-1ce0-44cd-935a-b58987e26570,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496891,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Aug 27 23:55:52.880: INFO: Logging kubelet events for node master3 Aug 27 23:55:52.883: INFO: Logging pods the kubelet thinks is on node master3 Aug 27 23:55:52.892: INFO: kube-proxy-8sxhm started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:52.892: INFO: Container kube-proxy ready: true, restart count 1 Aug 27 23:55:52.892: INFO: kube-flannel-fkz5d started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded) Aug 27 23:55:52.892: INFO: Init container install-cni ready: true, restart count 0 Aug 27 23:55:52.892: INFO: Container kube-flannel ready: true, restart count 1 Aug 27 23:55:52.892: INFO: dns-autoscaler-5b7b5c9b6f-54xch started at 2021-08-27 20:49:19 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:52.892: INFO: Container autoscaler ready: true, restart count 1 Aug 27 23:55:52.892: INFO: coredns-7677f9bb54-rxplt started at 2021-08-27 20:49:21 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:52.892: INFO: Container coredns ready: true, restart count 1 Aug 27 23:55:52.892: INFO: kube-controller-manager-master3 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:52.892: INFO: Container kube-controller-manager ready: true, restart count 2 Aug 27 23:55:52.892: INFO: kube-scheduler-master3 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:52.892: INFO: Container kube-scheduler ready: true, restart count 2 Aug 27 23:55:52.892: INFO: kube-multus-ds-amd64-wwcgv started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:52.892: INFO: Container kube-multus ready: true, restart count 1 Aug 27 23:55:52.892: INFO: node-exporter-d4m7q started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded) Aug 27 23:55:52.892: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 27 23:55:52.892: INFO: Container node-exporter ready: true, restart count 0 Aug 27 23:55:52.892: INFO: kube-apiserver-master3 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:52.892: INFO: Container kube-apiserver ready: true, restart count 0 W0827 23:55:52.906240 26 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. Aug 27 23:55:52.929: INFO: Latency metrics for node master3 Aug 27 23:55:52.929: INFO: Logging node info for node node1 Aug 27 23:55:52.931: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 e7a2481e-32db-4c83-bd9f-4a0687258e7a 63120 0 2021-08-27 20:48:09 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.36.2.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"56:c7:37:40:51:ca"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-08-27 20:48:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubeadm Update v1 2021-08-27 20:48:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-08-27 20:55:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-08-27 20:57:40 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-08-27 20:57:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269628928 0} {<nil>} 196552372Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884628480 0} {<nil>} 174692020Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:51:49 +0000 UTC,LastTransitionTime:2021-08-27 20:51:49 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-27 23:55:42 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-27 23:55:42 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-27 23:55:42 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-27 23:55:42 +0000 UTC,LastTransitionTime:2021-08-27 20:48:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:c1e38e80ea114a5f96601202301ce842,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:e769e86d-15c0-442c-a93b-bcc6c33ff1cd,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:331c6faa8b0d5ec72cf105e87d35df0a2f2baeec3d6217a51faa73f9460f937f localhost:30500/barometer-collectd:stable],SizeBytes:1238704157,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:a7cea43d9d2f67c38fbf0407786edbe660ee9072945f7bb272b55fd255e8eaca opnfv/barometer-collectd:stable],SizeBytes:1075746799,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:1003787960,},ContainerImage{Names:[localhost:30500/cmk@sha256:fd1487b0c07556a087eff669e70c501a704720dcd53ff75183593de6720585f2 cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[aquasec/kube-hunter@sha256:2be6820bc1d7e0f57193a9a27d5a3e16b2fd93c53747b03ce8ca48c6fc323781 aquasec/kube-hunter:0.3.1],SizeBytes:347611549,},ContainerImage{Names:[golang@sha256:db2475a1dbb2149508e5db31d7d77a75e6600d54be645f37681f03f2762169ba golang:alpine3.12],SizeBytes:301186719,},ContainerImage{Names:[sirot/netperf-latest@sha256:23929b922bb077cb341450fc1c90ae42e6f75da5d7ce61cd1880b08560a5ff85 sirot/netperf-latest:latest],SizeBytes:282025213,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:d7300ccf7ff3e9cea2111d275143b8050618bbc1d1ffe41f46286b1696261243 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44393508,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:de25c7fc6c4f3a27c7f0c2dff454e4671823a34d88abd533f210848d527e0fbb alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Aug 27 23:55:52.932: INFO: Logging kubelet events for node node1 Aug 27 23:55:52.934: INFO: Logging pods the kubelet thinks is on node node1 Aug 27 23:55:53.001: INFO: fail-once-local-nnfjf started at 2021-08-27 23:51:47 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:53.001: INFO: Container c ready: false, restart count 0 Aug 27 23:55:53.001: INFO: downward-api-1b4ad441-2b80-4fec-922c-5948ba3a182a started at 2021-08-27 23:54:01 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:53.001: INFO: Container dapi-container ready: false, restart count 0 Aug 27 23:55:53.001: INFO: kubernetes-dashboard-86c6f9df5b-c56fg started at 2021-08-27 20:49:21 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:53.001: INFO: Container kubernetes-dashboard ready: true, restart count 1 Aug 27 23:55:53.001: INFO: node-feature-discovery-worker-bd9kg started at 2021-08-27 20:55:06 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:53.002: INFO: Container nfd-worker ready: true, restart count 0 Aug 27 23:55:53.002: INFO: ss2-0 started at 2021-08-27 23:46:41 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:53.002: INFO: Container webserver ready: false, restart count 0 Aug 27 23:55:53.002: INFO: kube-proxy-pb5bl started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:53.002: INFO: Container kube-proxy ready: true, restart count 2 Aug 27 23:55:53.002: INFO: collectd-ccvwg started at 2021-08-27 21:04:15 +0000 UTC (0+3 container statuses recorded) Aug 27 23:55:53.002: INFO: Container collectd ready: true, restart count 0 Aug 27 23:55:53.002: INFO: Container collectd-exporter ready: true, restart count 0 Aug 27 23:55:53.002: INFO: Container rbac-proxy ready: true, restart count 0 Aug 27 23:55:53.002: INFO: prometheus-k8s-0 started at 2021-08-27 20:59:29 +0000 UTC (0+5 container statuses recorded) Aug 27 23:55:53.002: INFO: Container custom-metrics-apiserver ready: true, restart count 0 Aug 27 23:55:53.002: INFO: Container grafana ready: true, restart count 0 Aug 27 23:55:53.002: INFO: Container prometheus ready: true, restart count 1 Aug 27 23:55:53.002: INFO: Container prometheus-config-reloader ready: true, restart count 0 Aug 27 23:55:53.002: INFO: Container rules-configmap-reloader ready: true, restart count 0 Aug 27 23:55:53.002: INFO: pod-adoption started at 2021-08-27 23:50:54 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:53.002: INFO: Container pod-adoption ready: false, restart count 0 Aug 27 23:55:53.002: INFO: pod-projected-configmaps-c71ae7c5-e91d-4f64-9f49-5c817f06cec8 started at 2021-08-27 23:55:39 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:53.002: INFO: Container projected-configmap-volume-test ready: false, restart count 0 Aug 27 23:55:53.002: INFO: kube-flannel-ssxn7 started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded) Aug 27 23:55:53.002: INFO: Init container install-cni ready: true, restart count 0 Aug 27 23:55:53.002: INFO: Container kube-flannel ready: true, restart count 1 Aug 27 23:55:53.002: INFO: cmk-jw4m6 started at 2021-08-27 20:58:19 +0000 UTC (0+2 container statuses recorded) Aug 27 23:55:53.002: INFO: Container nodereport ready: true, restart count 0 Aug 27 23:55:53.002: INFO: Container reconcile ready: true, restart count 0 Aug 27 23:55:53.002: INFO: kube-multus-ds-amd64-nn7bl started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:53.002: INFO: Container kube-multus ready: true, restart count 1 Aug 27 23:55:53.002: INFO: kubernetes-metrics-scraper-678c97765c-gtp5x started at 2021-08-27 20:49:21 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:53.002: INFO: Container kubernetes-metrics-scraper ready: true, restart count 1 Aug 27 23:55:53.002: INFO: cmk-init-discover-node1-spg26 started at 2021-08-27 20:57:37 +0000 UTC (0+3 container statuses recorded) Aug 27 23:55:53.002: INFO: Container discover ready: false, restart count 0 Aug 27 23:55:53.002: INFO: Container init ready: false, restart count 0 Aug 27 23:55:53.002: INFO: Container install ready: false, restart count 0 Aug 27 23:55:53.002: INFO: nginx-proxy-node1 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:53.002: INFO: Container nginx-proxy ready: true, restart count 2 Aug 27 23:55:53.002: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-9lndx started at 2021-08-27 20:55:51 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:53.002: INFO: Container kube-sriovdp ready: true, restart count 0 Aug 27 23:55:53.002: INFO: node-exporter-4cvlq started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded) Aug 27 23:55:53.002: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 27 23:55:53.002: INFO: Container node-exporter ready: true, restart count 0 W0827 23:55:53.015505 26 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. Aug 27 23:55:53.044: INFO: Latency metrics for node node1 Aug 27 23:55:53.044: INFO: Logging node info for node node2 Aug 27 23:55:53.047: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 eb7da7c0-513f-4072-a078-ad3d24f88114 63195 0 2021-08-27 20:48:09 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.36.2.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"9a:88:9c:d1:39:68"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-08-27 20:48:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubeadm Update v1 2021-08-27 20:48:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-08-27 20:55:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-08-27 20:58:03 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-08-27 20:58:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269628928 0} {<nil>} 196552372Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884628480 0} {<nil>} 174692020Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:50:36 +0000 UTC,LastTransitionTime:2021-08-27 20:50:36 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-27 23:55:50 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-27 23:55:50 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-27 23:55:50 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-27 23:55:50 +0000 UTC,LastTransitionTime:2021-08-27 20:48:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:0ccfc2a4a9b7400c9ca53b5de0ca4970,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:26b02690-0814-4c92-9f6d-d315df796ce6,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:331c6faa8b0d5ec72cf105e87d35df0a2f2baeec3d6217a51faa73f9460f937f localhost:30500/barometer-collectd:stable],SizeBytes:1238704157,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[localhost:30500/cmk@sha256:fd1487b0c07556a087eff669e70c501a704720dcd53ff75183593de6720585f2 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:d7300ccf7ff3e9cea2111d275143b8050618bbc1d1ffe41f46286b1696261243 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44393508,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:ae72171f047a37ee5423e0692df7429830919af16e9d668ab0c80b723863d102 localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:cd98d1edca8e5e2e3ea42cbc463812483e5d069d10f0974ca9d484b5a7bd68db localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[aquasec/kube-bench@sha256:3544f6662feb73d36fdba35b17652e2fd73aae45bd4b60e76d7ab928220b3cc6 aquasec/kube-bench:0.3.1],SizeBytes:19301876,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Aug 27 23:55:53.048: INFO: Logging kubelet events for node node2 Aug 27 23:55:53.050: INFO: Logging pods the kubelet thinks is on node node2 Aug 27 23:55:53.062: INFO: kube-flannel-t9qv4 started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded) Aug 27 23:55:53.062: INFO: Init container install-cni ready: true, restart count 0 Aug 27 23:55:53.062: INFO: Container kube-flannel ready: true, restart count 2 Aug 27 23:55:53.062: INFO: cmk-init-discover-node2-l9qjd started at 2021-08-27 20:57:57 +0000 UTC (0+3 container statuses recorded) Aug 27 23:55:53.063: INFO: Container discover ready: false, restart count 0 Aug 27 23:55:53.063: INFO: Container init ready: false, restart count 0 Aug 27 23:55:53.063: INFO: Container install ready: false, restart count 0 Aug 27 23:55:53.063: INFO: node-feature-discovery-worker-54lfh started at 2021-08-27 20:55:06 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:53.063: INFO: Container nfd-worker ready: true, restart count 0 Aug 27 23:55:53.063: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4f962 started at 2021-08-27 20:55:51 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:53.063: INFO: Container kube-sriovdp ready: true, restart count 0 Aug 27 23:55:53.063: INFO: pod-init-87bf8589-6bea-4e5f-a081-a4af5b3a51cb started at 2021-08-27 23:55:52 +0000 UTC (2+1 container statuses recorded) Aug 27 23:55:53.063: INFO: Init container init1 ready: false, restart count 0 Aug 27 23:55:53.063: INFO: Init container init2 ready: false, restart count 0 Aug 27 23:55:53.063: INFO: Container run1 ready: false, restart count 0 Aug 27 23:55:53.063: INFO: ss-0 started at 2021-08-27 23:47:20 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:53.063: INFO: Container webserver ready: false, restart count 0 Aug 27 23:55:53.063: INFO: busybox-readonly-false-f7f9026d-2e01-458b-928e-8960287201ef started at 2021-08-27 23:50:47 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:53.063: INFO: Container busybox-readonly-false-f7f9026d-2e01-458b-928e-8960287201ef ready: false, restart count 0 Aug 27 23:55:53.063: INFO: fail-once-local-rthxs started at 2021-08-27 23:51:47 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:53.063: INFO: Container c ready: false, restart count 0 Aug 27 23:55:53.063: INFO: pod-configmaps-97befb70-76fa-4cc9-ae71-edd994d168b1 started at 2021-08-27 23:55:27 +0000 UTC (0+3 container statuses recorded) Aug 27 23:55:53.063: INFO: Container createcm-volume-test ready: true, restart count 0 Aug 27 23:55:53.063: INFO: Container delcm-volume-test ready: true, restart count 0 Aug 27 23:55:53.063: INFO: Container updcm-volume-test ready: true, restart count 0 Aug 27 23:55:53.063: INFO: nginx-proxy-node2 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:53.063: INFO: Container nginx-proxy ready: true, restart count 1 Aug 27 23:55:53.063: INFO: kube-proxy-r4q4t started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:53.063: INFO: Container kube-proxy ready: true, restart count 2 Aug 27 23:55:53.063: INFO: kube-multus-ds-amd64-tfffk started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:53.063: INFO: Container kube-multus ready: true, restart count 1 Aug 27 23:55:53.063: INFO: node-exporter-p6h5h started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded) Aug 27 23:55:53.063: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 27 23:55:53.063: INFO: Container node-exporter ready: true, restart count 0 Aug 27 23:55:53.063: INFO: cmk-fzjgr started at 2021-08-27 20:58:20 +0000 UTC (0+2 container statuses recorded) Aug 27 23:55:53.063: INFO: Container nodereport ready: true, restart count 0 Aug 27 23:55:53.063: INFO: Container reconcile ready: true, restart count 0 Aug 27 23:55:53.063: INFO: collectd-64dp2 started at 2021-08-27 21:04:15 +0000 UTC (0+3 container statuses recorded) Aug 27 23:55:53.063: INFO: Container collectd ready: true, restart count 0 Aug 27 23:55:53.063: INFO: Container collectd-exporter ready: true, restart count 0 Aug 27 23:55:53.063: INFO: Container rbac-proxy ready: true, restart count 0 Aug 27 23:55:53.063: INFO: pod-submit-remove-e0fe8833-506c-4fe2-a611-59e5057e1de3 started at 2021-08-27 23:55:30 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:53.063: INFO: Container nginx ready: false, restart count 0 Aug 27 23:55:53.063: INFO: pod-subpath-test-configmap-mdht started at 2021-08-27 23:55:45 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:53.063: INFO: Container test-container-subpath-configmap-mdht ready: true, restart count 0 Aug 27 23:55:53.063: INFO: cmk-webhook-6c9d5f8578-ndbx2 started at 2021-08-27 20:58:20 +0000 UTC (0+1 container statuses recorded) Aug 27 23:55:53.063: INFO: Container cmk-webhook ready: true, restart count 0 Aug 27 23:55:53.063: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-v99df started at 2021-08-27 21:02:08 +0000 UTC (0+2 container statuses recorded) Aug 27 23:55:53.063: INFO: Container tas-controller ready: true, restart count 0 Aug 27 23:55:53.063: INFO: Container tas-extender ready: true, restart count 0 W0827 23:55:53.076183 26 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. Aug 27 23:55:53.104: INFO: Latency metrics for node node2 Aug 27 23:55:53.104: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "var-expansion-6695" for this suite. /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 Aug 28 00:03:18.568: starting pod busybox-c9839ea8-82a2-4695-9414-b49b463dc2f6 in namespace container-probe-2345 Unexpected error: <*errors.errorString | 0xc0002801f0>: { s: "timed out waiting for the condition", } timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/container_probe.go:426 [BeforeEach] [k8s.io] Probing container /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client Aug 27 23:58:18.524: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename container-probe STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [k8s.io] Probing container /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/container_probe.go:54 [It] should *not* be restarted with a exec "cat /tmp/health" liveness probe [NodeConformance] [Conformance] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 STEP: Creating pod busybox-c9839ea8-82a2-4695-9414-b49b463dc2f6 in namespace container-probe-2345 Aug 28 00:03:18.568: FAIL: starting pod busybox-c9839ea8-82a2-4695-9414-b49b463dc2f6 in namespace container-probe-2345 Unexpected error: <*errors.errorString | 0xc0002801f0>: { s: "timed out waiting for the condition", } timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/common.RunLivenessTest(0xc0001ade40, 0xc005588000, 0x0, 0x37e11d6000) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/container_probe.go:426 +0x4b9 k8s.io/kubernetes/test/e2e/common.glob..func3.5() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/container_probe.go:146 +0x197 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc00180a780) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc00180a780) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc00180a780, 0x4dec428) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 STEP: deleting the pod [AfterEach] [k8s.io] Probing container /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "container-probe-2345". STEP: Found 9 events. Aug 28 00:03:18.581: INFO: At 2021-08-27 23:58:18 +0000 UTC - event for busybox-c9839ea8-82a2-4695-9414-b49b463dc2f6: {default-scheduler } Scheduled: Successfully assigned container-probe-2345/busybox-c9839ea8-82a2-4695-9414-b49b463dc2f6 to node2 Aug 28 00:03:18.582: INFO: At 2021-08-27 23:58:20 +0000 UTC - event for busybox-c9839ea8-82a2-4695-9414-b49b463dc2f6: {multus } AddedInterface: Add eth0 [10.244.4.96/24] Aug 28 00:03:18.582: INFO: At 2021-08-27 23:58:20 +0000 UTC - event for busybox-c9839ea8-82a2-4695-9414-b49b463dc2f6: {kubelet node2} Pulling: Pulling image "docker.io/library/busybox:1.29" Aug 28 00:03:18.582: INFO: At 2021-08-27 23:58:21 +0000 UTC - event for busybox-c9839ea8-82a2-4695-9414-b49b463dc2f6: {kubelet node2} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit Aug 28 00:03:18.582: INFO: At 2021-08-27 23:58:21 +0000 UTC - event for busybox-c9839ea8-82a2-4695-9414-b49b463dc2f6: {kubelet node2} Failed: Error: ErrImagePull Aug 28 00:03:18.582: INFO: At 2021-08-27 23:58:21 +0000 UTC - event for busybox-c9839ea8-82a2-4695-9414-b49b463dc2f6: {kubelet node2} SandboxChanged: Pod sandbox changed, it will be killed and re-created. Aug 28 00:03:18.582: INFO: At 2021-08-27 23:58:23 +0000 UTC - event for busybox-c9839ea8-82a2-4695-9414-b49b463dc2f6: {multus } AddedInterface: Add eth0 [10.244.4.98/24] Aug 28 00:03:18.582: INFO: At 2021-08-27 23:58:23 +0000 UTC - event for busybox-c9839ea8-82a2-4695-9414-b49b463dc2f6: {kubelet node2} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" Aug 28 00:03:18.582: INFO: At 2021-08-27 23:58:23 +0000 UTC - event for busybox-c9839ea8-82a2-4695-9414-b49b463dc2f6: {kubelet node2} Failed: Error: ImagePullBackOff Aug 28 00:03:18.584: INFO: POD NODE PHASE GRACE CONDITIONS Aug 28 00:03:18.584: INFO: Aug 28 00:03:18.588: INFO: Logging node info for node master1 Aug 28 00:03:18.591: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 3af53387-5aee-42c1-b0e7-644cf9161d48 67420 0 2021-08-27 20:46:13 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"92:f8:b6:72:e4:be"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-08-27 20:46:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-08-27 20:46:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {kube-controller-manager Update v1 2021-08-27 20:46:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:50:56 +0000 UTC,LastTransitionTime:2021-08-27 20:50:56 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 00:03:13 +0000 UTC,LastTransitionTime:2021-08-27 20:46:13 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 00:03:13 +0000 UTC,LastTransitionTime:2021-08-27 20:46:13 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 00:03:13 +0000 UTC,LastTransitionTime:2021-08-27 20:46:13 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 00:03:13 +0000 UTC,LastTransitionTime:2021-08-27 20:50:30 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:5d584023135a46ecb77596bf48ed7f2f,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:587cafa0-6de3-49f8-906e-06315a8ff104,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[sirot/netperf-latest@sha256:23929b922bb077cb341450fc1c90ae42e6f75da5d7ce61cd1880b08560a5ff85 sirot/netperf-latest:latest],SizeBytes:282025213,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:ae72171f047a37ee5423e0692df7429830919af16e9d668ab0c80b723863d102 tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:cd98d1edca8e5e2e3ea42cbc463812483e5d069d10f0974ca9d484b5a7bd68db tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[aquasec/kube-bench@sha256:3544f6662feb73d36fdba35b17652e2fd73aae45bd4b60e76d7ab928220b3cc6 aquasec/kube-bench:0.3.1],SizeBytes:19301876,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Aug 28 00:03:18.592: INFO: Logging kubelet events for node master1 Aug 28 00:03:18.594: INFO: Logging pods the kubelet thinks is on node master1 Aug 28 00:03:18.604: INFO: kube-scheduler-master1 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.604: INFO: Container kube-scheduler ready: true, restart count 0 Aug 28 00:03:18.604: INFO: kube-flannel-pp7vp started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded) Aug 28 00:03:18.604: INFO: Init container install-cni ready: true, restart count 0 Aug 28 00:03:18.604: INFO: Container kube-flannel ready: true, restart count 3 Aug 28 00:03:18.604: INFO: kube-multus-ds-amd64-sfr9k started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.604: INFO: Container kube-multus ready: true, restart count 1 Aug 28 00:03:18.604: INFO: docker-registry-docker-registry-56cbc7bc58-cthtt started at 2021-08-27 20:51:47 +0000 UTC (0+2 container statuses recorded) Aug 28 00:03:18.604: INFO: Container docker-registry ready: true, restart count 0 Aug 28 00:03:18.604: INFO: Container nginx ready: true, restart count 0 Aug 28 00:03:18.604: INFO: kube-apiserver-master1 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.604: INFO: Container kube-apiserver ready: true, restart count 0 Aug 28 00:03:18.604: INFO: kube-controller-manager-master1 started at 2021-08-27 20:54:27 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.604: INFO: Container kube-controller-manager ready: true, restart count 3 Aug 28 00:03:18.604: INFO: kube-proxy-rb5p6 started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.604: INFO: Container kube-proxy ready: true, restart count 1 Aug 28 00:03:18.604: INFO: coredns-7677f9bb54-dwtp5 started at 2021-08-27 20:49:16 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.604: INFO: Container coredns ready: true, restart count 1 Aug 28 00:03:18.604: INFO: node-exporter-z2ngr started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded) Aug 28 00:03:18.604: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 28 00:03:18.604: INFO: Container node-exporter ready: true, restart count 0 W0828 00:03:18.619092 26 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. Aug 28 00:03:18.644: INFO: Latency metrics for node master1 Aug 28 00:03:18.644: INFO: Logging node info for node master2 Aug 28 00:03:18.647: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 666e473f-d9e6-4c06-8b56-06474d788f70 67412 0 2021-08-27 20:46:51 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"42:86:ff:30:bd:4d"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-08-27 20:46:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {kubelet Update v1 2021-08-27 20:46:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-08-27 20:48:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-08-27 20:55:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:51:53 +0000 UTC,LastTransitionTime:2021-08-27 20:51:53 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 00:03:11 +0000 UTC,LastTransitionTime:2021-08-27 20:46:51 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 00:03:11 +0000 UTC,LastTransitionTime:2021-08-27 20:46:51 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 00:03:11 +0000 UTC,LastTransitionTime:2021-08-27 20:46:51 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 00:03:11 +0000 UTC,LastTransitionTime:2021-08-27 20:48:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:2835065974a64998811b9acd85de209b,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:309b9155-1e2a-4ebd-900f-bba5abfc3a5d,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[sirot/netperf-latest@sha256:23929b922bb077cb341450fc1c90ae42e6f75da5d7ce61cd1880b08560a5ff85 sirot/netperf-latest:latest],SizeBytes:282025213,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Aug 28 00:03:18.647: INFO: Logging kubelet events for node master2 Aug 28 00:03:18.649: INFO: Logging pods the kubelet thinks is on node master2 Aug 28 00:03:18.658: INFO: kube-apiserver-master2 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.658: INFO: Container kube-apiserver ready: true, restart count 0 Aug 28 00:03:18.658: INFO: kube-scheduler-master2 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.658: INFO: Container kube-scheduler ready: true, restart count 2 Aug 28 00:03:18.659: INFO: kube-proxy-b4mn9 started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.659: INFO: Container kube-proxy ready: true, restart count 1 Aug 28 00:03:18.659: INFO: node-feature-discovery-controller-5bf5c49849-zr9zd started at 2021-08-27 20:55:09 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.659: INFO: Container nfd-controller ready: true, restart count 0 Aug 28 00:03:18.659: INFO: kube-controller-manager-master2 started at 2021-08-27 20:51:01 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.659: INFO: Container kube-controller-manager ready: true, restart count 2 Aug 28 00:03:18.659: INFO: kube-flannel-4znnq started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded) Aug 28 00:03:18.659: INFO: Init container install-cni ready: true, restart count 2 Aug 28 00:03:18.659: INFO: Container kube-flannel ready: true, restart count 2 Aug 28 00:03:18.659: INFO: kube-multus-ds-amd64-4mgbk started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.659: INFO: Container kube-multus ready: true, restart count 1 Aug 28 00:03:18.659: INFO: prometheus-operator-5bb8cb9d8f-whr5p started at 2021-08-27 20:59:06 +0000 UTC (0+2 container statuses recorded) Aug 28 00:03:18.659: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 28 00:03:18.659: INFO: Container prometheus-operator ready: true, restart count 0 Aug 28 00:03:18.659: INFO: node-exporter-96jk5 started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded) Aug 28 00:03:18.659: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 28 00:03:18.659: INFO: Container node-exporter ready: true, restart count 0 W0828 00:03:18.673106 26 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. Aug 28 00:03:18.700: INFO: Latency metrics for node master2 Aug 28 00:03:18.700: INFO: Logging node info for node master3 Aug 28 00:03:18.703: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 620b6dba-f2c5-46e9-b2ff-d2f4197167d0 67405 0 2021-08-27 20:47:02 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ce:de:ea:c3:40:4f"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-08-27 20:47:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-08-27 20:47:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-08-27 20:48:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:51:44 +0000 UTC,LastTransitionTime:2021-08-27 20:51:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 00:03:09 +0000 UTC,LastTransitionTime:2021-08-27 20:47:02 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 00:03:09 +0000 UTC,LastTransitionTime:2021-08-27 20:47:02 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 00:03:09 +0000 UTC,LastTransitionTime:2021-08-27 20:47:02 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 00:03:09 +0000 UTC,LastTransitionTime:2021-08-27 20:48:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:37decbffe0e84048b5801289ad3be5bf,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:c96fe8b9-1ce0-44cd-935a-b58987e26570,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496891,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Aug 28 00:03:18.703: INFO: Logging kubelet events for node master3 Aug 28 00:03:18.706: INFO: Logging pods the kubelet thinks is on node master3 Aug 28 00:03:18.715: INFO: kube-controller-manager-master3 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.715: INFO: Container kube-controller-manager ready: true, restart count 2 Aug 28 00:03:18.715: INFO: kube-proxy-8sxhm started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.715: INFO: Container kube-proxy ready: true, restart count 1 Aug 28 00:03:18.715: INFO: kube-flannel-fkz5d started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded) Aug 28 00:03:18.715: INFO: Init container install-cni ready: true, restart count 0 Aug 28 00:03:18.715: INFO: Container kube-flannel ready: true, restart count 1 Aug 28 00:03:18.715: INFO: dns-autoscaler-5b7b5c9b6f-54xch started at 2021-08-27 20:49:19 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.715: INFO: Container autoscaler ready: true, restart count 1 Aug 28 00:03:18.715: INFO: coredns-7677f9bb54-rxplt started at 2021-08-27 20:49:21 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.715: INFO: Container coredns ready: true, restart count 1 Aug 28 00:03:18.715: INFO: kube-apiserver-master3 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.715: INFO: Container kube-apiserver ready: true, restart count 0 Aug 28 00:03:18.715: INFO: kube-scheduler-master3 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.715: INFO: Container kube-scheduler ready: true, restart count 2 Aug 28 00:03:18.715: INFO: kube-multus-ds-amd64-wwcgv started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.715: INFO: Container kube-multus ready: true, restart count 1 Aug 28 00:03:18.715: INFO: node-exporter-d4m7q started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded) Aug 28 00:03:18.715: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 28 00:03:18.715: INFO: Container node-exporter ready: true, restart count 0 W0828 00:03:18.729887 26 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. Aug 28 00:03:18.753: INFO: Latency metrics for node master3 Aug 28 00:03:18.753: INFO: Logging node info for node node1 Aug 28 00:03:18.756: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 e7a2481e-32db-4c83-bd9f-4a0687258e7a 67441 0 2021-08-27 20:48:09 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.36.2.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"56:c7:37:40:51:ca"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-08-27 20:48:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubeadm Update v1 2021-08-27 20:48:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-08-27 20:55:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-08-27 20:57:40 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-08-27 20:57:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269628928 0} {<nil>} 196552372Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884628480 0} {<nil>} 174692020Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:51:49 +0000 UTC,LastTransitionTime:2021-08-27 20:51:49 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 00:03:17 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 00:03:17 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 00:03:17 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 00:03:17 +0000 UTC,LastTransitionTime:2021-08-27 20:48:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:c1e38e80ea114a5f96601202301ce842,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:e769e86d-15c0-442c-a93b-bcc6c33ff1cd,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:331c6faa8b0d5ec72cf105e87d35df0a2f2baeec3d6217a51faa73f9460f937f localhost:30500/barometer-collectd:stable],SizeBytes:1238704157,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:a7cea43d9d2f67c38fbf0407786edbe660ee9072945f7bb272b55fd255e8eaca opnfv/barometer-collectd:stable],SizeBytes:1075746799,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:1003787960,},ContainerImage{Names:[localhost:30500/cmk@sha256:fd1487b0c07556a087eff669e70c501a704720dcd53ff75183593de6720585f2 cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[aquasec/kube-hunter@sha256:2be6820bc1d7e0f57193a9a27d5a3e16b2fd93c53747b03ce8ca48c6fc323781 aquasec/kube-hunter:0.3.1],SizeBytes:347611549,},ContainerImage{Names:[golang@sha256:db2475a1dbb2149508e5db31d7d77a75e6600d54be645f37681f03f2762169ba golang:alpine3.12],SizeBytes:301186719,},ContainerImage{Names:[sirot/netperf-latest@sha256:23929b922bb077cb341450fc1c90ae42e6f75da5d7ce61cd1880b08560a5ff85 sirot/netperf-latest:latest],SizeBytes:282025213,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:d7300ccf7ff3e9cea2111d275143b8050618bbc1d1ffe41f46286b1696261243 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44393508,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:de25c7fc6c4f3a27c7f0c2dff454e4671823a34d88abd533f210848d527e0fbb alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Aug 28 00:03:18.757: INFO: Logging kubelet events for node node1 Aug 28 00:03:18.760: INFO: Logging pods the kubelet thinks is on node node1 Aug 28 00:03:18.777: INFO: kube-flannel-ssxn7 started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded) Aug 28 00:03:18.777: INFO: Init container install-cni ready: true, restart count 0 Aug 28 00:03:18.777: INFO: Container kube-flannel ready: true, restart count 1 Aug 28 00:03:18.777: INFO: pod-configmaps-083121aa-d5da-435b-b71e-278033dca23d started at 2021-08-28 00:02:45 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.777: INFO: Container env-test ready: false, restart count 0 Aug 28 00:03:18.777: INFO: kube-multus-ds-amd64-nn7bl started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.777: INFO: Container kube-multus ready: true, restart count 1 Aug 28 00:03:18.777: INFO: kubernetes-metrics-scraper-678c97765c-gtp5x started at 2021-08-27 20:49:21 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.777: INFO: Container kubernetes-metrics-scraper ready: true, restart count 1 Aug 28 00:03:18.777: INFO: cmk-init-discover-node1-spg26 started at 2021-08-27 20:57:37 +0000 UTC (0+3 container statuses recorded) Aug 28 00:03:18.777: INFO: Container discover ready: false, restart count 0 Aug 28 00:03:18.777: INFO: Container init ready: false, restart count 0 Aug 28 00:03:18.777: INFO: Container install ready: false, restart count 0 Aug 28 00:03:18.777: INFO: cmk-jw4m6 started at 2021-08-27 20:58:19 +0000 UTC (0+2 container statuses recorded) Aug 28 00:03:18.777: INFO: Container nodereport ready: true, restart count 0 Aug 28 00:03:18.777: INFO: Container reconcile ready: true, restart count 0 Aug 28 00:03:18.777: INFO: nginx-proxy-node1 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.777: INFO: Container nginx-proxy ready: true, restart count 2 Aug 28 00:03:18.778: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-9lndx started at 2021-08-27 20:55:51 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.778: INFO: Container kube-sriovdp ready: true, restart count 0 Aug 28 00:03:18.778: INFO: node-exporter-4cvlq started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded) Aug 28 00:03:18.778: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 28 00:03:18.778: INFO: Container node-exporter ready: true, restart count 0 Aug 28 00:03:18.778: INFO: busybox-user-65534-4e812def-5fed-4cb2-833c-b1b74115ed6d started at 2021-08-28 00:02:08 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.778: INFO: Container busybox-user-65534-4e812def-5fed-4cb2-833c-b1b74115ed6d ready: false, restart count 0 Aug 28 00:03:18.778: INFO: ss-0 started at 2021-08-27 23:56:52 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.778: INFO: Container webserver ready: false, restart count 0 Aug 28 00:03:18.778: INFO: kubernetes-dashboard-86c6f9df5b-c56fg started at 2021-08-27 20:49:21 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.778: INFO: Container kubernetes-dashboard ready: true, restart count 1 Aug 28 00:03:18.778: INFO: node-feature-discovery-worker-bd9kg started at 2021-08-27 20:55:06 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.778: INFO: Container nfd-worker ready: true, restart count 0 Aug 28 00:03:18.778: INFO: fail-once-local-nnfjf started at 2021-08-27 23:51:47 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.778: INFO: Container c ready: false, restart count 0 Aug 28 00:03:18.778: INFO: kube-proxy-pb5bl started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.778: INFO: Container kube-proxy ready: true, restart count 2 Aug 28 00:03:18.778: INFO: collectd-ccvwg started at 2021-08-27 21:04:15 +0000 UTC (0+3 container statuses recorded) Aug 28 00:03:18.778: INFO: Container collectd ready: true, restart count 0 Aug 28 00:03:18.778: INFO: Container collectd-exporter ready: true, restart count 0 Aug 28 00:03:18.778: INFO: Container rbac-proxy ready: true, restart count 0 Aug 28 00:03:18.778: INFO: prometheus-k8s-0 started at 2021-08-27 20:59:29 +0000 UTC (0+5 container statuses recorded) Aug 28 00:03:18.778: INFO: Container custom-metrics-apiserver ready: true, restart count 0 Aug 28 00:03:18.778: INFO: Container grafana ready: true, restart count 0 Aug 28 00:03:18.778: INFO: Container prometheus ready: true, restart count 1 Aug 28 00:03:18.778: INFO: Container prometheus-config-reloader ready: true, restart count 0 Aug 28 00:03:18.778: INFO: Container rules-configmap-reloader ready: true, restart count 0 Aug 28 00:03:18.778: INFO: foo-wxzlz started at 2021-08-27 23:57:13 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.778: INFO: Container c ready: false, restart count 0 W0828 00:03:18.789325 26 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. Aug 28 00:03:18.823: INFO: Latency metrics for node node1 Aug 28 00:03:18.823: INFO: Logging node info for node node2 Aug 28 00:03:18.826: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 eb7da7c0-513f-4072-a078-ad3d24f88114 67413 0 2021-08-27 20:48:09 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.36.2.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"9a:88:9c:d1:39:68"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-08-27 20:48:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubeadm Update v1 2021-08-27 20:48:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-08-27 20:55:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-08-27 20:58:03 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-08-27 20:58:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269628928 0} {<nil>} 196552372Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884628480 0} {<nil>} 174692020Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:50:36 +0000 UTC,LastTransitionTime:2021-08-27 20:50:36 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 00:03:12 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 00:03:12 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 00:03:12 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 00:03:12 +0000 UTC,LastTransitionTime:2021-08-27 20:48:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:0ccfc2a4a9b7400c9ca53b5de0ca4970,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:26b02690-0814-4c92-9f6d-d315df796ce6,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:331c6faa8b0d5ec72cf105e87d35df0a2f2baeec3d6217a51faa73f9460f937f localhost:30500/barometer-collectd:stable],SizeBytes:1238704157,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[localhost:30500/cmk@sha256:fd1487b0c07556a087eff669e70c501a704720dcd53ff75183593de6720585f2 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:d7300ccf7ff3e9cea2111d275143b8050618bbc1d1ffe41f46286b1696261243 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44393508,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:ae72171f047a37ee5423e0692df7429830919af16e9d668ab0c80b723863d102 localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:cd98d1edca8e5e2e3ea42cbc463812483e5d069d10f0974ca9d484b5a7bd68db localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[aquasec/kube-bench@sha256:3544f6662feb73d36fdba35b17652e2fd73aae45bd4b60e76d7ab928220b3cc6 aquasec/kube-bench:0.3.1],SizeBytes:19301876,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Aug 28 00:03:18.826: INFO: Logging kubelet events for node node2 Aug 28 00:03:18.828: INFO: Logging pods the kubelet thinks is on node node2 Aug 28 00:03:18.891: INFO: collectd-64dp2 started at 2021-08-27 21:04:15 +0000 UTC (0+3 container statuses recorded) Aug 28 00:03:18.891: INFO: Container collectd ready: true, restart count 0 Aug 28 00:03:18.891: INFO: Container collectd-exporter ready: true, restart count 0 Aug 28 00:03:18.891: INFO: Container rbac-proxy ready: true, restart count 0 Aug 28 00:03:18.891: INFO: test-rollover-controller-nlwvt started at 2021-08-27 23:58:28 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.891: INFO: Container httpd ready: false, restart count 0 Aug 28 00:03:18.891: INFO: cmk-webhook-6c9d5f8578-ndbx2 started at 2021-08-27 20:58:20 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.891: INFO: Container cmk-webhook ready: true, restart count 0 Aug 28 00:03:18.891: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-v99df started at 2021-08-27 21:02:08 +0000 UTC (0+2 container statuses recorded) Aug 28 00:03:18.891: INFO: Container tas-controller ready: true, restart count 0 Aug 28 00:03:18.891: INFO: Container tas-extender ready: true, restart count 0 Aug 28 00:03:18.891: INFO: kube-flannel-t9qv4 started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded) Aug 28 00:03:18.891: INFO: Init container install-cni ready: true, restart count 0 Aug 28 00:03:18.891: INFO: Container kube-flannel ready: true, restart count 2 Aug 28 00:03:18.891: INFO: cmk-init-discover-node2-l9qjd started at 2021-08-27 20:57:57 +0000 UTC (0+3 container statuses recorded) Aug 28 00:03:18.891: INFO: Container discover ready: false, restart count 0 Aug 28 00:03:18.891: INFO: Container init ready: false, restart count 0 Aug 28 00:03:18.891: INFO: Container install ready: false, restart count 0 Aug 28 00:03:18.891: INFO: projected-volume-e94e9a41-f528-4051-a982-fe427a6060f1 started at 2021-08-28 00:00:37 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.891: INFO: Container projected-all-volume-test ready: false, restart count 0 Aug 28 00:03:18.891: INFO: downward-api-7982f5fb-4c85-430d-8f8c-8c165783418e started at 2021-08-28 00:00:53 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.891: INFO: Container dapi-container ready: false, restart count 0 Aug 28 00:03:18.891: INFO: node-feature-discovery-worker-54lfh started at 2021-08-27 20:55:06 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.891: INFO: Container nfd-worker ready: true, restart count 0 Aug 28 00:03:18.891: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4f962 started at 2021-08-27 20:55:51 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.891: INFO: Container kube-sriovdp ready: true, restart count 0 Aug 28 00:03:18.891: INFO: foo-qhqsk started at 2021-08-27 23:57:13 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.891: INFO: Container c ready: false, restart count 0 Aug 28 00:03:18.891: INFO: fail-once-local-rthxs started at 2021-08-27 23:51:47 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.891: INFO: Container c ready: false, restart count 0 Aug 28 00:03:18.891: INFO: nginx-proxy-node2 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.891: INFO: Container nginx-proxy ready: true, restart count 1 Aug 28 00:03:18.891: INFO: kube-proxy-r4q4t started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.891: INFO: Container kube-proxy ready: true, restart count 2 Aug 28 00:03:18.891: INFO: kube-multus-ds-amd64-tfffk started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.891: INFO: Container kube-multus ready: true, restart count 1 Aug 28 00:03:18.891: INFO: node-exporter-p6h5h started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded) Aug 28 00:03:18.891: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 28 00:03:18.891: INFO: Container node-exporter ready: true, restart count 0 Aug 28 00:03:18.891: INFO: cmk-fzjgr started at 2021-08-27 20:58:20 +0000 UTC (0+2 container statuses recorded) Aug 28 00:03:18.891: INFO: Container nodereport ready: true, restart count 0 Aug 28 00:03:18.891: INFO: Container reconcile ready: true, restart count 0 Aug 28 00:03:18.891: INFO: test-pod started at 2021-08-28 00:01:27 +0000 UTC (0+1 container statuses recorded) Aug 28 00:03:18.891: INFO: Container webserver ready: false, restart count 0 W0828 00:03:18.905449 26 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. Aug 28 00:03:18.946: INFO: Latency metrics for node node2 Aug 28 00:03:18.946: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "container-probe-2345" for this suite. /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 Aug 28 00:09:34.115: wait for pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a" to succeed Expected success, but got an error: <*errors.errorString | 0xc002644300>: { s: "Gave up after waiting 5m0s for pod \"busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a\" to be \"Succeeded or Failed\"", } Gave up after waiting 5m0s for pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a" to be "Succeeded or Failed" /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/pods.go:212 [BeforeEach] [k8s.io] Security Context /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client Aug 28 00:04:33.560: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename security-context-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [k8s.io] Security Context /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/security_context.go:41 [It] should run the container as unprivileged when false [LinuxOnly] [NodeConformance] [Conformance] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 Aug 28 00:04:33.592: INFO: Waiting up to 5m0s for pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a" in namespace "security-context-test-3721" to be "Succeeded or Failed" Aug 28 00:04:33.594: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 2.397053ms Aug 28 00:04:35.597: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 2.00526851s Aug 28 00:04:37.600: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 4.007790028s Aug 28 00:04:39.603: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 6.011258456s Aug 28 00:04:41.607: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 8.015261928s Aug 28 00:04:43.612: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 10.019432328s Aug 28 00:04:45.615: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 12.022941099s Aug 28 00:04:47.619: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 14.027102682s Aug 28 00:04:49.623: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 16.030846646s Aug 28 00:04:51.627: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 18.035195951s Aug 28 00:04:53.633: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 20.040728568s Aug 28 00:04:55.636: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 22.044063676s Aug 28 00:04:57.639: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 24.047185231s Aug 28 00:04:59.642: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 26.050060047s Aug 28 00:05:01.645: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 28.053150429s Aug 28 00:05:03.649: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 30.05718863s Aug 28 00:05:05.653: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 32.061401784s Aug 28 00:05:07.658: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 34.065940181s Aug 28 00:05:09.662: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 36.069548708s Aug 28 00:05:11.666: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 38.073772496s Aug 28 00:05:13.669: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 40.077304997s Aug 28 00:05:15.673: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 42.080808985s Aug 28 00:05:17.676: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 44.084421009s Aug 28 00:05:19.681: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 46.088438327s Aug 28 00:05:21.684: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 48.091810344s Aug 28 00:05:23.686: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 50.094305531s Aug 28 00:05:25.690: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 52.09774822s Aug 28 00:05:27.695: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 54.102516064s Aug 28 00:05:29.698: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 56.106258483s Aug 28 00:05:31.701: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 58.109369465s Aug 28 00:05:33.707: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.11448108s Aug 28 00:05:35.710: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.117679752s Aug 28 00:05:37.714: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.121458265s Aug 28 00:05:39.717: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.124873527s Aug 28 00:05:41.720: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.128055231s Aug 28 00:05:43.723: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.130830459s Aug 28 00:05:45.726: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.133460346s Aug 28 00:05:47.730: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.13789826s Aug 28 00:05:49.733: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.140688306s Aug 28 00:05:51.736: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.143660217s Aug 28 00:05:53.739: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.146467196s Aug 28 00:05:55.741: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.149396619s Aug 28 00:05:57.746: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.153440786s Aug 28 00:05:59.748: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.156319279s Aug 28 00:06:01.751: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.158892208s Aug 28 00:06:03.754: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.161587735s Aug 28 00:06:05.757: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.164735661s Aug 28 00:06:07.759: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.167095052s Aug 28 00:06:09.763: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.170439183s Aug 28 00:06:11.766: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.174217299s Aug 28 00:06:13.769: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.176839261s Aug 28 00:06:15.773: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.180696011s Aug 28 00:06:17.777: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.184648404s Aug 28 00:06:19.781: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.188744794s Aug 28 00:06:21.785: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.192535616s Aug 28 00:06:23.788: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.196009958s Aug 28 00:06:25.791: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.199173602s Aug 28 00:06:27.794: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.202163619s Aug 28 00:06:29.798: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.205986516s Aug 28 00:06:31.802: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.209683879s Aug 28 00:06:33.805: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.213009564s Aug 28 00:06:35.809: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.216694439s Aug 28 00:06:37.816: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.223789606s Aug 28 00:06:39.819: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.226576906s Aug 28 00:06:41.822: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.229686841s Aug 28 00:06:43.825: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.232474628s Aug 28 00:06:45.829: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.236456314s Aug 28 00:06:47.831: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.239130693s Aug 28 00:06:49.835: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.242447466s Aug 28 00:06:51.838: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.245844218s Aug 28 00:06:53.841: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.248733936s Aug 28 00:06:55.844: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.251703535s Aug 28 00:06:57.847: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.254839611s Aug 28 00:06:59.850: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.257761772s Aug 28 00:07:01.852: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.260260688s Aug 28 00:07:03.858: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.265557623s Aug 28 00:07:05.862: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.269505773s Aug 28 00:07:07.865: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.273098352s Aug 28 00:07:09.868: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.27628585s Aug 28 00:07:11.871: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.279258834s Aug 28 00:07:13.875: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.283017196s Aug 28 00:07:15.879: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.287079774s Aug 28 00:07:17.884: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.292142057s Aug 28 00:07:19.889: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.296867649s Aug 28 00:07:21.892: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.300304806s Aug 28 00:07:23.896: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.304381104s Aug 28 00:07:25.899: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.307113805s Aug 28 00:07:27.902: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.310202162s Aug 28 00:07:29.906: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.314209129s Aug 28 00:07:31.910: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.31813863s Aug 28 00:07:33.914: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.321592197s Aug 28 00:07:35.916: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.324228972s Aug 28 00:07:37.919: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.326812982s Aug 28 00:07:39.922: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.329613125s Aug 28 00:07:41.925: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.333304016s Aug 28 00:07:43.928: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.336359639s Aug 28 00:07:45.932: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.340129707s Aug 28 00:07:47.935: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.343226187s Aug 28 00:07:49.940: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.347568467s Aug 28 00:07:51.943: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.350840329s Aug 28 00:07:53.947: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.354752041s Aug 28 00:07:55.951: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.359171259s Aug 28 00:07:57.954: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.361746737s Aug 28 00:07:59.957: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.364647237s Aug 28 00:08:01.959: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.366768981s Aug 28 00:08:03.961: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.36938467s Aug 28 00:08:05.964: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.372422552s Aug 28 00:08:07.968: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.376079513s Aug 28 00:08:09.971: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.379029402s Aug 28 00:08:11.974: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.382250663s Aug 28 00:08:13.977: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.384740629s Aug 28 00:08:15.979: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.387350021s Aug 28 00:08:17.983: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.39063619s Aug 28 00:08:19.986: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.394019593s Aug 28 00:08:21.990: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.397901374s Aug 28 00:08:23.993: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.401129394s Aug 28 00:08:25.997: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.404547219s Aug 28 00:08:28.001: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.408842107s Aug 28 00:08:30.005: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.413219539s Aug 28 00:08:32.009: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.417258001s Aug 28 00:08:34.012: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.420344886s Aug 28 00:08:36.016: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.424283952s Aug 28 00:08:38.021: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.428588107s Aug 28 00:08:40.024: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.432278974s Aug 28 00:08:42.028: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.435541586s Aug 28 00:08:44.031: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.439289486s Aug 28 00:08:46.034: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.442110708s Aug 28 00:08:48.039: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.446474283s Aug 28 00:08:50.043: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.450879377s Aug 28 00:08:52.046: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.453516351s Aug 28 00:08:54.049: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.457213817s Aug 28 00:08:56.053: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.461039514s Aug 28 00:08:58.058: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.465910683s Aug 28 00:09:00.061: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.469078938s Aug 28 00:09:02.064: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.472398279s Aug 28 00:09:04.067: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.474910295s Aug 28 00:09:06.070: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.477438576s Aug 28 00:09:08.073: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.481318472s Aug 28 00:09:10.076: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.483993973s Aug 28 00:09:12.079: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.48727809s Aug 28 00:09:14.083: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.490465588s Aug 28 00:09:16.085: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.493251711s Aug 28 00:09:18.088: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.496083753s Aug 28 00:09:20.093: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.500550659s Aug 28 00:09:22.096: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.503603886s Aug 28 00:09:24.098: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.506412243s Aug 28 00:09:26.102: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.510408119s Aug 28 00:09:28.106: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.514186876s Aug 28 00:09:30.109: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.517160194s Aug 28 00:09:32.113: INFO: Pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.520934169s Aug 28 00:09:34.114: FAIL: wait for pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a" to succeed Expected success, but got an error: <*errors.errorString | 0xc002644300>: { s: "Gave up after waiting 5m0s for pod \"busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a\" to be \"Succeeded or Failed\"", } Gave up after waiting 5m0s for pod "busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a" to be "Succeeded or Failed" Full Stack Trace k8s.io/kubernetes/test/e2e/framework.(*PodClient).WaitForSuccess(0xc003b526e0, 0xc002077340, 0x3d, 0x45d964b800) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/pods.go:212 +0x2bb k8s.io/kubernetes/test/e2e/common.glob..func29.5.2(0x0, 0x0, 0xc003b526e0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/security_context.go:255 +0x1ec k8s.io/kubernetes/test/e2e/common.glob..func29.5.3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/security_context.go:265 +0x48 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc00180a780) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc00180a780) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc00180a780, 0x4dec428) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [k8s.io] Security Context /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "security-context-test-3721". STEP: Found 7 events. Aug 28 00:09:34.120: INFO: At 2021-08-28 00:04:33 +0000 UTC - event for busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a: {default-scheduler } Scheduled: Successfully assigned security-context-test-3721/busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a to node2 Aug 28 00:09:34.120: INFO: At 2021-08-28 00:04:34 +0000 UTC - event for busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a: {multus } AddedInterface: Add eth0 [10.244.4.115/24] Aug 28 00:09:34.120: INFO: At 2021-08-28 00:04:34 +0000 UTC - event for busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a: {kubelet node2} Pulling: Pulling image "docker.io/library/busybox:1.29" Aug 28 00:09:34.120: INFO: At 2021-08-28 00:04:36 +0000 UTC - event for busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a: {kubelet node2} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit Aug 28 00:09:34.120: INFO: At 2021-08-28 00:04:36 +0000 UTC - event for busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a: {kubelet node2} Failed: Error: ErrImagePull Aug 28 00:09:34.120: INFO: At 2021-08-28 00:04:36 +0000 UTC - event for busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a: {kubelet node2} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" Aug 28 00:09:34.120: INFO: At 2021-08-28 00:04:36 +0000 UTC - event for busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a: {kubelet node2} Failed: Error: ImagePullBackOff Aug 28 00:09:34.123: INFO: POD NODE PHASE GRACE CONDITIONS Aug 28 00:09:34.123: INFO: busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a node2 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:04:33 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:04:33 +0000 UTC ContainersNotReady containers with unready status: [busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:04:33 +0000 UTC ContainersNotReady containers with unready status: [busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:04:33 +0000 UTC }] Aug 28 00:09:34.123: INFO: Aug 28 00:09:34.127: INFO: Logging node info for node master1 Aug 28 00:09:34.130: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 3af53387-5aee-42c1-b0e7-644cf9161d48 70496 0 2021-08-27 20:46:13 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"92:f8:b6:72:e4:be"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-08-27 20:46:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-08-27 20:46:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {kube-controller-manager Update v1 2021-08-27 20:46:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:50:56 +0000 UTC,LastTransitionTime:2021-08-27 20:50:56 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 00:09:24 +0000 UTC,LastTransitionTime:2021-08-27 20:46:13 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 00:09:24 +0000 UTC,LastTransitionTime:2021-08-27 20:46:13 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 00:09:24 +0000 UTC,LastTransitionTime:2021-08-27 20:46:13 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 00:09:24 +0000 UTC,LastTransitionTime:2021-08-27 20:50:30 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:5d584023135a46ecb77596bf48ed7f2f,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:587cafa0-6de3-49f8-906e-06315a8ff104,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[sirot/netperf-latest@sha256:23929b922bb077cb341450fc1c90ae42e6f75da5d7ce61cd1880b08560a5ff85 sirot/netperf-latest:latest],SizeBytes:282025213,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:ae72171f047a37ee5423e0692df7429830919af16e9d668ab0c80b723863d102 tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:cd98d1edca8e5e2e3ea42cbc463812483e5d069d10f0974ca9d484b5a7bd68db tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[aquasec/kube-bench@sha256:3544f6662feb73d36fdba35b17652e2fd73aae45bd4b60e76d7ab928220b3cc6 aquasec/kube-bench:0.3.1],SizeBytes:19301876,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Aug 28 00:09:34.131: INFO: Logging kubelet events for node master1 Aug 28 00:09:34.133: INFO: Logging pods the kubelet thinks is on node master1 Aug 28 00:09:34.142: INFO: kube-scheduler-master1 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.142: INFO: Container kube-scheduler ready: true, restart count 0 Aug 28 00:09:34.142: INFO: kube-flannel-pp7vp started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded) Aug 28 00:09:34.142: INFO: Init container install-cni ready: true, restart count 0 Aug 28 00:09:34.142: INFO: Container kube-flannel ready: true, restart count 3 Aug 28 00:09:34.142: INFO: kube-multus-ds-amd64-sfr9k started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.142: INFO: Container kube-multus ready: true, restart count 1 Aug 28 00:09:34.142: INFO: docker-registry-docker-registry-56cbc7bc58-cthtt started at 2021-08-27 20:51:47 +0000 UTC (0+2 container statuses recorded) Aug 28 00:09:34.142: INFO: Container docker-registry ready: true, restart count 0 Aug 28 00:09:34.142: INFO: Container nginx ready: true, restart count 0 Aug 28 00:09:34.142: INFO: kube-apiserver-master1 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.142: INFO: Container kube-apiserver ready: true, restart count 0 Aug 28 00:09:34.142: INFO: kube-controller-manager-master1 started at 2021-08-27 20:54:27 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.142: INFO: Container kube-controller-manager ready: true, restart count 3 Aug 28 00:09:34.142: INFO: kube-proxy-rb5p6 started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.142: INFO: Container kube-proxy ready: true, restart count 1 Aug 28 00:09:34.142: INFO: coredns-7677f9bb54-dwtp5 started at 2021-08-27 20:49:16 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.142: INFO: Container coredns ready: true, restart count 1 Aug 28 00:09:34.142: INFO: node-exporter-z2ngr started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded) Aug 28 00:09:34.142: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 28 00:09:34.142: INFO: Container node-exporter ready: true, restart count 0 W0828 00:09:34.156540 26 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. Aug 28 00:09:34.182: INFO: Latency metrics for node master1 Aug 28 00:09:34.182: INFO: Logging node info for node master2 Aug 28 00:09:34.184: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 666e473f-d9e6-4c06-8b56-06474d788f70 70527 0 2021-08-27 20:46:51 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"42:86:ff:30:bd:4d"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-08-27 20:46:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {kubelet Update v1 2021-08-27 20:46:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-08-27 20:48:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-08-27 20:55:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:51:53 +0000 UTC,LastTransitionTime:2021-08-27 20:51:53 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 00:09:32 +0000 UTC,LastTransitionTime:2021-08-27 20:46:51 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 00:09:32 +0000 UTC,LastTransitionTime:2021-08-27 20:46:51 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 00:09:32 +0000 UTC,LastTransitionTime:2021-08-27 20:46:51 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 00:09:32 +0000 UTC,LastTransitionTime:2021-08-27 20:48:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:2835065974a64998811b9acd85de209b,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:309b9155-1e2a-4ebd-900f-bba5abfc3a5d,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[sirot/netperf-latest@sha256:23929b922bb077cb341450fc1c90ae42e6f75da5d7ce61cd1880b08560a5ff85 sirot/netperf-latest:latest],SizeBytes:282025213,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Aug 28 00:09:34.185: INFO: Logging kubelet events for node master2 Aug 28 00:09:34.187: INFO: Logging pods the kubelet thinks is on node master2 Aug 28 00:09:34.196: INFO: kube-controller-manager-master2 started at 2021-08-27 20:51:01 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.196: INFO: Container kube-controller-manager ready: true, restart count 2 Aug 28 00:09:34.196: INFO: kube-flannel-4znnq started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded) Aug 28 00:09:34.196: INFO: Init container install-cni ready: true, restart count 2 Aug 28 00:09:34.196: INFO: Container kube-flannel ready: true, restart count 2 Aug 28 00:09:34.196: INFO: kube-multus-ds-amd64-4mgbk started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.196: INFO: Container kube-multus ready: true, restart count 1 Aug 28 00:09:34.196: INFO: prometheus-operator-5bb8cb9d8f-whr5p started at 2021-08-27 20:59:06 +0000 UTC (0+2 container statuses recorded) Aug 28 00:09:34.196: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 28 00:09:34.196: INFO: Container prometheus-operator ready: true, restart count 0 Aug 28 00:09:34.196: INFO: node-exporter-96jk5 started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded) Aug 28 00:09:34.196: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 28 00:09:34.196: INFO: Container node-exporter ready: true, restart count 0 Aug 28 00:09:34.196: INFO: kube-apiserver-master2 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.196: INFO: Container kube-apiserver ready: true, restart count 0 Aug 28 00:09:34.196: INFO: kube-scheduler-master2 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.196: INFO: Container kube-scheduler ready: true, restart count 2 Aug 28 00:09:34.196: INFO: kube-proxy-b4mn9 started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.196: INFO: Container kube-proxy ready: true, restart count 1 Aug 28 00:09:34.196: INFO: node-feature-discovery-controller-5bf5c49849-zr9zd started at 2021-08-27 20:55:09 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.196: INFO: Container nfd-controller ready: true, restart count 0 W0828 00:09:34.208986 26 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. Aug 28 00:09:34.233: INFO: Latency metrics for node master2 Aug 28 00:09:34.233: INFO: Logging node info for node master3 Aug 28 00:09:34.235: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 620b6dba-f2c5-46e9-b2ff-d2f4197167d0 70521 0 2021-08-27 20:47:02 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ce:de:ea:c3:40:4f"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-08-27 20:47:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-08-27 20:47:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-08-27 20:48:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:51:44 +0000 UTC,LastTransitionTime:2021-08-27 20:51:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 00:09:31 +0000 UTC,LastTransitionTime:2021-08-27 20:47:02 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 00:09:31 +0000 UTC,LastTransitionTime:2021-08-27 20:47:02 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 00:09:31 +0000 UTC,LastTransitionTime:2021-08-27 20:47:02 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 00:09:31 +0000 UTC,LastTransitionTime:2021-08-27 20:48:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:37decbffe0e84048b5801289ad3be5bf,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:c96fe8b9-1ce0-44cd-935a-b58987e26570,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496891,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Aug 28 00:09:34.236: INFO: Logging kubelet events for node master3 Aug 28 00:09:34.238: INFO: Logging pods the kubelet thinks is on node master3 Aug 28 00:09:34.247: INFO: dns-autoscaler-5b7b5c9b6f-54xch started at 2021-08-27 20:49:19 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.247: INFO: Container autoscaler ready: true, restart count 1 Aug 28 00:09:34.247: INFO: coredns-7677f9bb54-rxplt started at 2021-08-27 20:49:21 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.247: INFO: Container coredns ready: true, restart count 1 Aug 28 00:09:34.247: INFO: kube-controller-manager-master3 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.247: INFO: Container kube-controller-manager ready: true, restart count 2 Aug 28 00:09:34.247: INFO: kube-proxy-8sxhm started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.247: INFO: Container kube-proxy ready: true, restart count 1 Aug 28 00:09:34.247: INFO: kube-flannel-fkz5d started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded) Aug 28 00:09:34.247: INFO: Init container install-cni ready: true, restart count 0 Aug 28 00:09:34.247: INFO: Container kube-flannel ready: true, restart count 1 Aug 28 00:09:34.247: INFO: node-exporter-d4m7q started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded) Aug 28 00:09:34.247: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 28 00:09:34.247: INFO: Container node-exporter ready: true, restart count 0 Aug 28 00:09:34.247: INFO: kube-apiserver-master3 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.247: INFO: Container kube-apiserver ready: true, restart count 0 Aug 28 00:09:34.247: INFO: kube-scheduler-master3 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.247: INFO: Container kube-scheduler ready: true, restart count 2 Aug 28 00:09:34.247: INFO: kube-multus-ds-amd64-wwcgv started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.247: INFO: Container kube-multus ready: true, restart count 1 W0828 00:09:34.260619 26 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. Aug 28 00:09:34.292: INFO: Latency metrics for node master3 Aug 28 00:09:34.292: INFO: Logging node info for node node1 Aug 28 00:09:34.296: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 e7a2481e-32db-4c83-bd9f-4a0687258e7a 70512 0 2021-08-27 20:48:09 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.36.2.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"56:c7:37:40:51:ca"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-08-27 20:48:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubeadm Update v1 2021-08-27 20:48:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-08-27 20:55:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-08-27 20:57:40 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-08-27 20:57:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269628928 0} {<nil>} 196552372Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884628480 0} {<nil>} 174692020Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:51:49 +0000 UTC,LastTransitionTime:2021-08-27 20:51:49 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 00:09:29 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 00:09:29 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 00:09:29 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 00:09:29 +0000 UTC,LastTransitionTime:2021-08-27 20:48:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:c1e38e80ea114a5f96601202301ce842,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:e769e86d-15c0-442c-a93b-bcc6c33ff1cd,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:331c6faa8b0d5ec72cf105e87d35df0a2f2baeec3d6217a51faa73f9460f937f localhost:30500/barometer-collectd:stable],SizeBytes:1238704157,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:a7cea43d9d2f67c38fbf0407786edbe660ee9072945f7bb272b55fd255e8eaca opnfv/barometer-collectd:stable],SizeBytes:1075746799,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:1003787960,},ContainerImage{Names:[localhost:30500/cmk@sha256:fd1487b0c07556a087eff669e70c501a704720dcd53ff75183593de6720585f2 cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[aquasec/kube-hunter@sha256:2be6820bc1d7e0f57193a9a27d5a3e16b2fd93c53747b03ce8ca48c6fc323781 aquasec/kube-hunter:0.3.1],SizeBytes:347611549,},ContainerImage{Names:[golang@sha256:db2475a1dbb2149508e5db31d7d77a75e6600d54be645f37681f03f2762169ba golang:alpine3.12],SizeBytes:301186719,},ContainerImage{Names:[sirot/netperf-latest@sha256:23929b922bb077cb341450fc1c90ae42e6f75da5d7ce61cd1880b08560a5ff85 sirot/netperf-latest:latest],SizeBytes:282025213,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:d7300ccf7ff3e9cea2111d275143b8050618bbc1d1ffe41f46286b1696261243 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44393508,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:de25c7fc6c4f3a27c7f0c2dff454e4671823a34d88abd533f210848d527e0fbb alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Aug 28 00:09:34.297: INFO: Logging kubelet events for node node1 Aug 28 00:09:34.299: INFO: Logging pods the kubelet thinks is on node node1 Aug 28 00:09:34.315: INFO: kube-multus-ds-amd64-nn7bl started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.315: INFO: Container kube-multus ready: true, restart count 1 Aug 28 00:09:34.315: INFO: kubernetes-metrics-scraper-678c97765c-gtp5x started at 2021-08-27 20:49:21 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.315: INFO: Container kubernetes-metrics-scraper ready: true, restart count 1 Aug 28 00:09:34.315: INFO: cmk-init-discover-node1-spg26 started at 2021-08-27 20:57:37 +0000 UTC (0+3 container statuses recorded) Aug 28 00:09:34.315: INFO: Container discover ready: false, restart count 0 Aug 28 00:09:34.315: INFO: Container init ready: false, restart count 0 Aug 28 00:09:34.315: INFO: Container install ready: false, restart count 0 Aug 28 00:09:34.315: INFO: cmk-jw4m6 started at 2021-08-27 20:58:19 +0000 UTC (0+2 container statuses recorded) Aug 28 00:09:34.315: INFO: Container nodereport ready: true, restart count 0 Aug 28 00:09:34.315: INFO: Container reconcile ready: true, restart count 0 Aug 28 00:09:34.315: INFO: nginx-proxy-node1 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.315: INFO: Container nginx-proxy ready: true, restart count 2 Aug 28 00:09:34.315: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-9lndx started at 2021-08-27 20:55:51 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.315: INFO: Container kube-sriovdp ready: true, restart count 0 Aug 28 00:09:34.315: INFO: node-exporter-4cvlq started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded) Aug 28 00:09:34.315: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 28 00:09:34.315: INFO: Container node-exporter ready: true, restart count 0 Aug 28 00:09:34.315: INFO: var-expansion-5297e858-64f9-4f37-8f24-be27ca5c0f6e started at 2021-08-28 00:07:03 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.315: INFO: Container dapi-container ready: false, restart count 0 Aug 28 00:09:34.315: INFO: kubernetes-dashboard-86c6f9df5b-c56fg started at 2021-08-27 20:49:21 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.315: INFO: Container kubernetes-dashboard ready: true, restart count 1 Aug 28 00:09:34.315: INFO: node-feature-discovery-worker-bd9kg started at 2021-08-27 20:55:06 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.315: INFO: Container nfd-worker ready: true, restart count 0 Aug 28 00:09:34.315: INFO: pod-configmaps-b72f648b-a248-483c-a3c6-74fd04030e7a started at 2021-08-28 00:07:20 +0000 UTC (0+2 container statuses recorded) Aug 28 00:09:34.315: INFO: Container configmap-volume-binary-test ready: false, restart count 0 Aug 28 00:09:34.315: INFO: Container configmap-volume-data-test ready: true, restart count 0 Aug 28 00:09:34.315: INFO: kube-proxy-pb5bl started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.315: INFO: Container kube-proxy ready: true, restart count 2 Aug 28 00:09:34.315: INFO: collectd-ccvwg started at 2021-08-27 21:04:15 +0000 UTC (0+3 container statuses recorded) Aug 28 00:09:34.315: INFO: Container collectd ready: true, restart count 0 Aug 28 00:09:34.315: INFO: Container collectd-exporter ready: true, restart count 0 Aug 28 00:09:34.315: INFO: Container rbac-proxy ready: true, restart count 0 Aug 28 00:09:34.315: INFO: prometheus-k8s-0 started at 2021-08-27 20:59:29 +0000 UTC (0+5 container statuses recorded) Aug 28 00:09:34.315: INFO: Container custom-metrics-apiserver ready: true, restart count 0 Aug 28 00:09:34.315: INFO: Container grafana ready: true, restart count 0 Aug 28 00:09:34.315: INFO: Container prometheus ready: true, restart count 1 Aug 28 00:09:34.315: INFO: Container prometheus-config-reloader ready: true, restart count 0 Aug 28 00:09:34.315: INFO: Container rules-configmap-reloader ready: true, restart count 0 Aug 28 00:09:34.315: INFO: adopt-release-s6jg4 started at 2021-08-28 00:07:35 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.315: INFO: Container c ready: false, restart count 0 Aug 28 00:09:34.315: INFO: foo-wxzlz started at 2021-08-27 23:57:13 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.315: INFO: Container c ready: false, restart count 0 Aug 28 00:09:34.315: INFO: kube-flannel-ssxn7 started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded) Aug 28 00:09:34.315: INFO: Init container install-cni ready: true, restart count 0 Aug 28 00:09:34.315: INFO: Container kube-flannel ready: true, restart count 1 W0828 00:09:34.327905 26 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. Aug 28 00:09:34.374: INFO: Latency metrics for node node1 Aug 28 00:09:34.374: INFO: Logging node info for node node2 Aug 28 00:09:34.378: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 eb7da7c0-513f-4072-a078-ad3d24f88114 70533 0 2021-08-27 20:48:09 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.36.2.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"9a:88:9c:d1:39:68"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-08-27 20:48:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubeadm Update v1 2021-08-27 20:48:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-08-27 20:55:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-08-27 20:58:03 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-08-27 20:58:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269628928 0} {<nil>} 196552372Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884628480 0} {<nil>} 174692020Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:50:36 +0000 UTC,LastTransitionTime:2021-08-27 20:50:36 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 00:09:33 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 00:09:33 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 00:09:33 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 00:09:33 +0000 UTC,LastTransitionTime:2021-08-27 20:48:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:0ccfc2a4a9b7400c9ca53b5de0ca4970,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:26b02690-0814-4c92-9f6d-d315df796ce6,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:331c6faa8b0d5ec72cf105e87d35df0a2f2baeec3d6217a51faa73f9460f937f localhost:30500/barometer-collectd:stable],SizeBytes:1238704157,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[localhost:30500/cmk@sha256:fd1487b0c07556a087eff669e70c501a704720dcd53ff75183593de6720585f2 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:d7300ccf7ff3e9cea2111d275143b8050618bbc1d1ffe41f46286b1696261243 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44393508,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:ae72171f047a37ee5423e0692df7429830919af16e9d668ab0c80b723863d102 localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:cd98d1edca8e5e2e3ea42cbc463812483e5d069d10f0974ca9d484b5a7bd68db localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[aquasec/kube-bench@sha256:3544f6662feb73d36fdba35b17652e2fd73aae45bd4b60e76d7ab928220b3cc6 aquasec/kube-bench:0.3.1],SizeBytes:19301876,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Aug 28 00:09:34.378: INFO: Logging kubelet events for node node2 Aug 28 00:09:34.380: INFO: Logging pods the kubelet thinks is on node node2 Aug 28 00:09:34.395: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4f962 started at 2021-08-27 20:55:51 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.395: INFO: Container kube-sriovdp ready: true, restart count 0 Aug 28 00:09:34.395: INFO: foo-qhqsk started at 2021-08-27 23:57:13 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.395: INFO: Container c ready: false, restart count 0 Aug 28 00:09:34.395: INFO: node-feature-discovery-worker-54lfh started at 2021-08-27 20:55:06 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.395: INFO: Container nfd-worker ready: true, restart count 0 Aug 28 00:09:34.395: INFO: kube-proxy-r4q4t started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.395: INFO: Container kube-proxy ready: true, restart count 2 Aug 28 00:09:34.395: INFO: kube-multus-ds-amd64-tfffk started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.395: INFO: Container kube-multus ready: true, restart count 1 Aug 28 00:09:34.395: INFO: node-exporter-p6h5h started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded) Aug 28 00:09:34.395: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 28 00:09:34.395: INFO: Container node-exporter ready: true, restart count 0 Aug 28 00:09:34.395: INFO: busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a started at 2021-08-28 00:04:33 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.395: INFO: Container busybox-privileged-false-90b48784-5dd2-427b-b747-6afc064ddd7a ready: false, restart count 0 Aug 28 00:09:34.395: INFO: nginx-proxy-node2 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.395: INFO: Container nginx-proxy ready: true, restart count 1 Aug 28 00:09:34.395: INFO: adopt-release-thh76 started at 2021-08-28 00:07:35 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.395: INFO: Container c ready: false, restart count 0 Aug 28 00:09:34.395: INFO: busybox-readonly-fs4ff1ee3a-ef4c-4c58-b1a3-272770e65c5f started at 2021-08-28 00:05:57 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.395: INFO: Container busybox-readonly-fs4ff1ee3a-ef4c-4c58-b1a3-272770e65c5f ready: false, restart count 0 Aug 28 00:09:34.395: INFO: termination-message-containerf5986240-8f49-4b9c-b2e3-7702c8d9ced1 started at 2021-08-28 00:07:48 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.395: INFO: Container termination-message-container ready: false, restart count 0 Aug 28 00:09:34.395: INFO: cmk-fzjgr started at 2021-08-27 20:58:20 +0000 UTC (0+2 container statuses recorded) Aug 28 00:09:34.395: INFO: Container nodereport ready: true, restart count 0 Aug 28 00:09:34.395: INFO: Container reconcile ready: true, restart count 0 Aug 28 00:09:34.395: INFO: pod-update-activedeadlineseconds-1307540e-0d68-4e42-be9b-24f48a89a9da started at 2021-08-28 00:08:29 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.395: INFO: Container nginx ready: false, restart count 0 Aug 28 00:09:34.395: INFO: collectd-64dp2 started at 2021-08-27 21:04:15 +0000 UTC (0+3 container statuses recorded) Aug 28 00:09:34.395: INFO: Container collectd ready: true, restart count 0 Aug 28 00:09:34.395: INFO: Container collectd-exporter ready: true, restart count 0 Aug 28 00:09:34.395: INFO: Container rbac-proxy ready: true, restart count 0 Aug 28 00:09:34.395: INFO: test-webserver-ddaa0c4d-b240-4406-a2ed-9089f76bb382 started at 2021-08-28 00:07:09 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.395: INFO: Container test-webserver ready: true, restart count 0 Aug 28 00:09:34.395: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-v99df started at 2021-08-27 21:02:08 +0000 UTC (0+2 container statuses recorded) Aug 28 00:09:34.395: INFO: Container tas-controller ready: true, restart count 0 Aug 28 00:09:34.395: INFO: Container tas-extender ready: true, restart count 0 Aug 28 00:09:34.395: INFO: cmk-webhook-6c9d5f8578-ndbx2 started at 2021-08-27 20:58:20 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.395: INFO: Container cmk-webhook ready: true, restart count 0 Aug 28 00:09:34.395: INFO: cmk-init-discover-node2-l9qjd started at 2021-08-27 20:57:57 +0000 UTC (0+3 container statuses recorded) Aug 28 00:09:34.395: INFO: Container discover ready: false, restart count 0 Aug 28 00:09:34.395: INFO: Container init ready: false, restart count 0 Aug 28 00:09:34.395: INFO: Container install ready: false, restart count 0 Aug 28 00:09:34.395: INFO: liveness-0d0e184a-16e9-40a6-a9e6-c55adc80a942 started at 2021-08-28 00:06:02 +0000 UTC (0+1 container statuses recorded) Aug 28 00:09:34.395: INFO: Container liveness ready: true, restart count 0 Aug 28 00:09:34.395: INFO: kube-flannel-t9qv4 started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded) Aug 28 00:09:34.395: INFO: Init container install-cni ready: true, restart count 0 Aug 28 00:09:34.395: INFO: Container kube-flannel ready: true, restart count 2 W0828 00:09:34.410661 26 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. Aug 28 00:09:34.441: INFO: Latency metrics for node node2 Aug 28 00:09:34.441: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "security-context-test-3721" for this suite. /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 Aug 28 00:14:44.790: error in waiting for pods to come up: failed to wait for pods running: [timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition] Unexpected error: <*errors.errorString | 0xc00222a6b0>: { s: "failed to wait for pods running: [timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition]", } failed to wait for pods running: [timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition] occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/apps/deployment.go:736 [BeforeEach] [sig-apps] Deployment /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client Aug 28 00:09:42.744: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename deployment STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-apps] Deployment /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/apps/deployment.go:78 [It] deployment should support proportional scaling [Conformance] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 Aug 28 00:09:42.762: INFO: Creating deployment "webserver-deployment" Aug 28 00:09:42.768: INFO: Waiting for observed generation 1 Aug 28 00:09:44.773: INFO: Waiting for all required pods to come up Aug 28 00:09:44.777: INFO: Pod name httpd: Found 10 pods out of 10 STEP: ensuring each pod is running Aug 28 00:14:44.790: FAIL: error in waiting for pods to come up: failed to wait for pods running: [timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition] Unexpected error: <*errors.errorString | 0xc00222a6b0>: { s: "failed to wait for pods running: [timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition]", } failed to wait for pods running: [timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition timed out waiting for the condition] occurred Full Stack Trace k8s.io/kubernetes/test/e2e/apps.testProportionalScalingDeployment(0xc0002f6f20) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/apps/deployment.go:736 +0x7ef k8s.io/kubernetes/test/e2e/apps.glob..func4.10() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/apps/deployment.go:135 +0x2a k8s.io/kubernetes/test/e2e.RunE2ETests(0xc00180a780) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc00180a780) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc00180a780, 0x4dec428) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [sig-apps] Deployment /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/apps/deployment.go:72 Aug 28 00:14:44.795: INFO: Deployment "webserver-deployment": &Deployment{ObjectMeta:{webserver-deployment deployment-4063 /apis/apps/v1/namespaces/deployment-4063/deployments/webserver-deployment 72e451e3-d2d1-4258-9fe3-58cccb1ba959 70693 1 2021-08-28 00:09:42 +0000 UTC <nil> <nil> map[name:httpd] map[deployment.kubernetes.io/revision:1] [] [] [{e2e.test Update apps/v1 2021-08-28 00:09:42 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{".":{},"f:name":{}}},"f:spec":{"f:progressDeadlineSeconds":{},"f:replicas":{},"f:revisionHistoryLimit":{},"f:selector":{"f:matchLabels":{".":{},"f:name":{}}},"f:strategy":{"f:rollingUpdate":{".":{},"f:maxSurge":{},"f:maxUnavailable":{}},"f:type":{}},"f:template":{"f:metadata":{"f:labels":{".":{},"f:name":{}}},"f:spec":{"f:containers":{"k:{\"name\":\"httpd\"}":{".":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:securityContext":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{}}},"f:dnsPolicy":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:terminationGracePeriodSeconds":{}}}}}} {kube-controller-manager Update apps/v1 2021-08-28 00:09:42 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:deployment.kubernetes.io/revision":{}}},"f:status":{"f:conditions":{".":{},"k:{\"type\":\"Available\"}":{".":{},"f:lastTransitionTime":{},"f:lastUpdateTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Progressing\"}":{".":{},"f:lastTransitionTime":{},"f:lastUpdateTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:observedGeneration":{},"f:replicas":{},"f:unavailableReplicas":{},"f:updatedReplicas":{}}}}]},Spec:DeploymentSpec{Replicas:*10,Selector:&v1.LabelSelector{MatchLabels:map[string]string{name: httpd,},MatchExpressions:[]LabelSelectorRequirement{},},Template:{{ 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[name:httpd] map[] [] [] []} {[] [] [{httpd docker.io/library/httpd:2.4.38-alpine [] [] [] [] [] {map[] map[]} [] [] nil nil nil nil /dev/termination-log File IfNotPresent SecurityContext{Capabilities:nil,Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,} false false false}] [] Always 0xc002c899b8 <nil> ClusterFirst map[] <nil> false false false <nil> &PodSecurityContext{SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,SupplementalGroups:[],FSGroup:nil,RunAsGroup:nil,Sysctls:[]Sysctl{},WindowsOptions:nil,FSGroupChangePolicy:nil,SeccompProfile:nil,} [] nil default-scheduler [] [] <nil> nil [] <nil> <nil> <nil> map[] [] <nil>}},Strategy:DeploymentStrategy{Type:RollingUpdate,RollingUpdate:&RollingUpdateDeployment{MaxUnavailable:2,MaxSurge:3,},},MinReadySeconds:0,RevisionHistoryLimit:*10,Paused:false,ProgressDeadlineSeconds:*600,},Status:DeploymentStatus{ObservedGeneration:1,Replicas:10,UpdatedReplicas:10,AvailableReplicas:0,UnavailableReplicas:10,Conditions:[]DeploymentCondition{DeploymentCondition{Type:Available,Status:False,Reason:MinimumReplicasUnavailable,Message:Deployment does not have minimum availability.,LastUpdateTime:2021-08-28 00:09:42 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,},DeploymentCondition{Type:Progressing,Status:True,Reason:ReplicaSetUpdated,Message:ReplicaSet "webserver-deployment-dd94f59b7" is progressing.,LastUpdateTime:2021-08-28 00:09:42 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,},},ReadyReplicas:0,CollisionCount:nil,},} Aug 28 00:14:44.797: INFO: New ReplicaSet "webserver-deployment-dd94f59b7" of Deployment "webserver-deployment": &ReplicaSet{ObjectMeta:{webserver-deployment-dd94f59b7 deployment-4063 /apis/apps/v1/namespaces/deployment-4063/replicasets/webserver-deployment-dd94f59b7 7163f4e0-9f27-4d87-a33d-f37e159ea03a 70691 1 2021-08-28 00:09:42 +0000 UTC <nil> <nil> map[name:httpd pod-template-hash:dd94f59b7] map[deployment.kubernetes.io/desired-replicas:10 deployment.kubernetes.io/max-replicas:13 deployment.kubernetes.io/revision:1] [{apps/v1 Deployment webserver-deployment 72e451e3-d2d1-4258-9fe3-58cccb1ba959 0xc002c89e37 0xc002c89e38}] [] [{kube-controller-manager Update apps/v1 2021-08-28 00:09:42 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:deployment.kubernetes.io/desired-replicas":{},"f:deployment.kubernetes.io/max-replicas":{},"f:deployment.kubernetes.io/revision":{}},"f:labels":{".":{},"f:name":{},"f:pod-template-hash":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"72e451e3-d2d1-4258-9fe3-58cccb1ba959\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:replicas":{},"f:selector":{"f:matchLabels":{".":{},"f:name":{},"f:pod-template-hash":{}}},"f:template":{"f:metadata":{"f:labels":{".":{},"f:name":{},"f:pod-template-hash":{}}},"f:spec":{"f:containers":{"k:{\"name\":\"httpd\"}":{".":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:securityContext":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{}}},"f:dnsPolicy":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:terminationGracePeriodSeconds":{}}}},"f:status":{"f:fullyLabeledReplicas":{},"f:observedGeneration":{},"f:replicas":{}}}}]},Spec:ReplicaSetSpec{Replicas:*10,Selector:&v1.LabelSelector{MatchLabels:map[string]string{name: httpd,pod-template-hash: dd94f59b7,},MatchExpressions:[]LabelSelectorRequirement{},},Template:{{ 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[name:httpd pod-template-hash:dd94f59b7] map[] [] [] []} {[] [] [{httpd docker.io/library/httpd:2.4.38-alpine [] [] [] [] [] {map[] map[]} [] [] nil nil nil nil /dev/termination-log File IfNotPresent SecurityContext{Capabilities:nil,Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,} false false false}] [] Always 0xc002c89ea8 <nil> ClusterFirst map[] <nil> false false false <nil> &PodSecurityContext{SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,SupplementalGroups:[],FSGroup:nil,RunAsGroup:nil,Sysctls:[]Sysctl{},WindowsOptions:nil,FSGroupChangePolicy:nil,SeccompProfile:nil,} [] nil default-scheduler [] [] <nil> nil [] <nil> <nil> <nil> map[] [] <nil>}},MinReadySeconds:0,},Status:ReplicaSetStatus{Replicas:10,FullyLabeledReplicas:10,ObservedGeneration:1,ReadyReplicas:0,AvailableReplicas:0,Conditions:[]ReplicaSetCondition{},},} Aug 28 00:14:44.801: INFO: Pod "webserver-deployment-dd94f59b7-2z9dr" is not available: &Pod{ObjectMeta:{webserver-deployment-dd94f59b7-2z9dr webserver-deployment-dd94f59b7- deployment-4063 /api/v1/namespaces/deployment-4063/pods/webserver-deployment-dd94f59b7-2z9dr c3257b64-fe4b-493c-bac4-ada3569f827d 74949 0 2021-08-28 00:09:42 +0000 UTC <nil> <nil> map[name:httpd pod-template-hash:dd94f59b7] map[k8s.v1.cni.cncf.io/network-status:[{ "name": "default-cni-network", "interface": "eth0", "ips": [ "10.244.4.143" ], "mac": "ee:83:f6:1c:18:2f", "default": true, "dns": {} }] k8s.v1.cni.cncf.io/networks-status:[{ "name": "default-cni-network", "interface": "eth0", "ips": [ "10.244.4.143" ], "mac": "ee:83:f6:1c:18:2f", "default": true, "dns": {} }] kubernetes.io/psp:collectd] [{apps/v1 ReplicaSet webserver-deployment-dd94f59b7 7163f4e0-9f27-4d87-a33d-f37e159ea03a 0xc0004a530f 0xc0004a5320}] [] [{kube-controller-manager Update v1 2021-08-28 00:09:42 +0000 UTC FieldsV1 {"f:metadata":{"f:generateName":{},"f:labels":{".":{},"f:name":{},"f:pod-template-hash":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"7163f4e0-9f27-4d87-a33d-f37e159ea03a\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:containers":{"k:{\"name\":\"httpd\"}":{".":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:securityContext":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:terminationGracePeriodSeconds":{}}}} {kubelet Update v1 2021-08-28 00:09:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"ContainersReady\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Initialized\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:containerStatuses":{},"f:hostIP":{},"f:podIP":{},"f:podIPs":{".":{},"k:{\"ip\":\"10.244.4.143\"}":{".":{},"f:ip":{}}},"f:startTime":{}}}} {multus Update v1 2021-08-28 00:09:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:k8s.v1.cni.cncf.io/network-status":{},"f:k8s.v1.cni.cncf.io/networks-status":{}}}}}]},Spec:PodSpec{Volumes:[]Volume{Volume{Name:default-token-d5hhc,VolumeSource:VolumeSource{HostPath:nil,EmptyDir:nil,GCEPersistentDisk:nil,AWSElasticBlockStore:nil,GitRepo:nil,Secret:&SecretVolumeSource{SecretName:default-token-d5hhc,Items:[]KeyToPath{},DefaultMode:*420,Optional:nil,},NFS:nil,ISCSI:nil,Glusterfs:nil,PersistentVolumeClaim:nil,RBD:nil,FlexVolume:nil,Cinder:nil,CephFS:nil,Flocker:nil,DownwardAPI:nil,FC:nil,AzureFile:nil,ConfigMap:nil,VsphereVolume:nil,Quobyte:nil,AzureDisk:nil,PhotonPersistentDisk:nil,PortworxVolume:nil,ScaleIO:nil,Projected:nil,StorageOS:nil,CSI:nil,Ephemeral:nil,},},},Containers:[]Container{Container{Name:httpd,Image:docker.io/library/httpd:2.4.38-alpine,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:default-token-d5hhc,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:nil,Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,},},RestartPolicy:Always,TerminationGracePeriodSeconds:*0,ActiveDeadlineSeconds:nil,DNSPolicy:ClusterFirst,NodeSelector:map[string]string{},ServiceAccountName:default,DeprecatedServiceAccount:default,NodeName:node2,HostNetwork:false,HostPID:false,HostIPC:false,SecurityContext:&PodSecurityContext{SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,SupplementalGroups:[],FSGroup:nil,RunAsGroup:nil,Sysctls:[]Sysctl{},WindowsOptions:nil,FSGroupChangePolicy:nil,SeccompProfile:nil,},ImagePullSecrets:[]LocalObjectReference{},Hostname:,Subdomain:,Affinity:nil,SchedulerName:default-scheduler,InitContainers:[]Container{},AutomountServiceAccountToken:nil,Tolerations:[]Toleration{Toleration{Key:node.kubernetes.io/not-ready,Operator:Exists,Value:,Effect:NoExecute,TolerationSeconds:*300,},Toleration{Key:node.kubernetes.io/unreachable,Operator:Exists,Value:,Effect:NoExecute,TolerationSeconds:*300,},},HostAliases:[]HostAlias{},PriorityClassName:,Priority:*0,DNSConfig:nil,ShareProcessNamespace:nil,ReadinessGates:[]PodReadinessGate{},RuntimeClassName:nil,EnableServiceLinks:*true,PreemptionPolicy:*PreemptLowerPriority,Overhead:ResourceList{},TopologySpreadConstraints:[]TopologySpreadConstraint{},EphemeralContainers:[]EphemeralContainer{},SetHostnameAsFQDN:nil,},Status:PodStatus{Phase:Pending,Conditions:[]PodCondition{PodCondition{Type:Initialized,Status:True,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:,Message:,},PodCondition{Type:Ready,Status:False,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:ContainersNotReady,Message:containers with unready status: [httpd],},PodCondition{Type:ContainersReady,Status:False,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:ContainersNotReady,Message:containers with unready status: [httpd],},PodCondition{Type:PodScheduled,Status:True,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:,Message:,},},Message:,Reason:,HostIP:10.10.190.208,PodIP:10.244.4.143,StartTime:2021-08-28 00:09:42 +0000 UTC,ContainerStatuses:[]ContainerStatus{ContainerStatus{Name:httpd,State:ContainerState{Waiting:&ContainerStateWaiting{Reason:ImagePullBackOff,Message:Back-off pulling image "docker.io/library/httpd:2.4.38-alpine",},Running:nil,Terminated:nil,},LastTerminationState:ContainerState{Waiting:nil,Running:nil,Terminated:nil,},Ready:false,RestartCount:0,Image:docker.io/library/httpd:2.4.38-alpine,ImageID:,ContainerID:,Started:*false,},},QOSClass:BestEffort,InitContainerStatuses:[]ContainerStatus{},NominatedNodeName:,PodIPs:[]PodIP{PodIP{IP:10.244.4.143,},},EphemeralContainerStatuses:[]ContainerStatus{},},} Aug 28 00:14:44.801: INFO: Pod "webserver-deployment-dd94f59b7-4jn4k" is not available: &Pod{ObjectMeta:{webserver-deployment-dd94f59b7-4jn4k webserver-deployment-dd94f59b7- deployment-4063 /api/v1/namespaces/deployment-4063/pods/webserver-deployment-dd94f59b7-4jn4k 6b845d79-850d-44f8-a2c8-9c593337de0e 74924 0 2021-08-28 00:09:42 +0000 UTC <nil> <nil> map[name:httpd pod-template-hash:dd94f59b7] map[k8s.v1.cni.cncf.io/network-status:[{ "name": "default-cni-network", "interface": "eth0", "ips": [ "10.244.4.142" ], "mac": "82:b8:14:90:92:d6", "default": true, "dns": {} }] k8s.v1.cni.cncf.io/networks-status:[{ "name": "default-cni-network", "interface": "eth0", "ips": [ "10.244.4.142" ], "mac": "82:b8:14:90:92:d6", "default": true, "dns": {} }] kubernetes.io/psp:collectd] [{apps/v1 ReplicaSet webserver-deployment-dd94f59b7 7163f4e0-9f27-4d87-a33d-f37e159ea03a 0xc0004a594f 0xc0004a5960}] [] [{kube-controller-manager Update v1 2021-08-28 00:09:42 +0000 UTC FieldsV1 {"f:metadata":{"f:generateName":{},"f:labels":{".":{},"f:name":{},"f:pod-template-hash":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"7163f4e0-9f27-4d87-a33d-f37e159ea03a\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:containers":{"k:{\"name\":\"httpd\"}":{".":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:securityContext":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:terminationGracePeriodSeconds":{}}}} {kubelet Update v1 2021-08-28 00:09:55 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"ContainersReady\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Initialized\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:containerStatuses":{},"f:hostIP":{},"f:podIP":{},"f:podIPs":{".":{},"k:{\"ip\":\"10.244.4.142\"}":{".":{},"f:ip":{}}},"f:startTime":{}}}} {multus Update v1 2021-08-28 00:09:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:k8s.v1.cni.cncf.io/network-status":{},"f:k8s.v1.cni.cncf.io/networks-status":{}}}}}]},Spec:PodSpec{Volumes:[]Volume{Volume{Name:default-token-d5hhc,VolumeSource:VolumeSource{HostPath:nil,EmptyDir:nil,GCEPersistentDisk:nil,AWSElasticBlockStore:nil,GitRepo:nil,Secret:&SecretVolumeSource{SecretName:default-token-d5hhc,Items:[]KeyToPath{},DefaultMode:*420,Optional:nil,},NFS:nil,ISCSI:nil,Glusterfs:nil,PersistentVolumeClaim:nil,RBD:nil,FlexVolume:nil,Cinder:nil,CephFS:nil,Flocker:nil,DownwardAPI:nil,FC:nil,AzureFile:nil,ConfigMap:nil,VsphereVolume:nil,Quobyte:nil,AzureDisk:nil,PhotonPersistentDisk:nil,PortworxVolume:nil,ScaleIO:nil,Projected:nil,StorageOS:nil,CSI:nil,Ephemeral:nil,},},},Containers:[]Container{Container{Name:httpd,Image:docker.io/library/httpd:2.4.38-alpine,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:default-token-d5hhc,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:nil,Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,},},RestartPolicy:Always,TerminationGracePeriodSeconds:*0,ActiveDeadlineSeconds:nil,DNSPolicy:ClusterFirst,NodeSelector:map[string]string{},ServiceAccountName:default,DeprecatedServiceAccount:default,NodeName:node2,HostNetwork:false,HostPID:false,HostIPC:false,SecurityContext:&PodSecurityContext{SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,SupplementalGroups:[],FSGroup:nil,RunAsGroup:nil,Sysctls:[]Sysctl{},WindowsOptions:nil,FSGroupChangePolicy:nil,SeccompProfile:nil,},ImagePullSecrets:[]LocalObjectReference{},Hostname:,Subdomain:,Affinity:nil,SchedulerName:default-scheduler,InitContainers:[]Container{},AutomountServiceAccountToken:nil,Tolerations:[]Toleration{Toleration{Key:node.kubernetes.io/not-ready,Operator:Exists,Value:,Effect:NoExecute,TolerationSeconds:*300,},Toleration{Key:node.kubernetes.io/unreachable,Operator:Exists,Value:,Effect:NoExecute,TolerationSeconds:*300,},},HostAliases:[]HostAlias{},PriorityClassName:,Priority:*0,DNSConfig:nil,ShareProcessNamespace:nil,ReadinessGates:[]PodReadinessGate{},RuntimeClassName:nil,EnableServiceLinks:*true,PreemptionPolicy:*PreemptLowerPriority,Overhead:ResourceList{},TopologySpreadConstraints:[]TopologySpreadConstraint{},EphemeralContainers:[]EphemeralContainer{},SetHostnameAsFQDN:nil,},Status:PodStatus{Phase:Pending,Conditions:[]PodCondition{PodCondition{Type:Initialized,Status:True,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:,Message:,},PodCondition{Type:Ready,Status:False,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:ContainersNotReady,Message:containers with unready status: [httpd],},PodCondition{Type:ContainersReady,Status:False,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:ContainersNotReady,Message:containers with unready status: [httpd],},PodCondition{Type:PodScheduled,Status:True,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:,Message:,},},Message:,Reason:,HostIP:10.10.190.208,PodIP:10.244.4.142,StartTime:2021-08-28 00:09:42 +0000 UTC,ContainerStatuses:[]ContainerStatus{ContainerStatus{Name:httpd,State:ContainerState{Waiting:&ContainerStateWaiting{Reason:ImagePullBackOff,Message:Back-off pulling image "docker.io/library/httpd:2.4.38-alpine",},Running:nil,Terminated:nil,},LastTerminationState:ContainerState{Waiting:nil,Running:nil,Terminated:nil,},Ready:false,RestartCount:0,Image:docker.io/library/httpd:2.4.38-alpine,ImageID:,ContainerID:,Started:*false,},},QOSClass:BestEffort,InitContainerStatuses:[]ContainerStatus{},NominatedNodeName:,PodIPs:[]PodIP{PodIP{IP:10.244.4.142,},},EphemeralContainerStatuses:[]ContainerStatus{},},} Aug 28 00:14:44.802: INFO: Pod "webserver-deployment-dd94f59b7-56q7x" is not available: &Pod{ObjectMeta:{webserver-deployment-dd94f59b7-56q7x webserver-deployment-dd94f59b7- deployment-4063 /api/v1/namespaces/deployment-4063/pods/webserver-deployment-dd94f59b7-56q7x e9a14787-27b1-46f7-b031-bfafaa072720 74757 0 2021-08-28 00:09:42 +0000 UTC <nil> <nil> map[name:httpd pod-template-hash:dd94f59b7] map[k8s.v1.cni.cncf.io/network-status:[{ "name": "default-cni-network", "interface": "eth0", "ips": [ "10.244.3.89" ], "mac": "76:21:20:7f:72:1b", "default": true, "dns": {} }] k8s.v1.cni.cncf.io/networks-status:[{ "name": "default-cni-network", "interface": "eth0", "ips": [ "10.244.3.89" ], "mac": "76:21:20:7f:72:1b", "default": true, "dns": {} }] kubernetes.io/psp:collectd] [{apps/v1 ReplicaSet webserver-deployment-dd94f59b7 7163f4e0-9f27-4d87-a33d-f37e159ea03a 0xc0004a5dbf 0xc0004a5de0}] [] [{kube-controller-manager Update v1 2021-08-28 00:09:42 +0000 UTC FieldsV1 {"f:metadata":{"f:generateName":{},"f:labels":{".":{},"f:name":{},"f:pod-template-hash":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"7163f4e0-9f27-4d87-a33d-f37e159ea03a\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:containers":{"k:{\"name\":\"httpd\"}":{".":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:securityContext":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:terminationGracePeriodSeconds":{}}}} {multus Update v1 2021-08-28 00:09:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:k8s.v1.cni.cncf.io/network-status":{},"f:k8s.v1.cni.cncf.io/networks-status":{}}}}} {kubelet Update v1 2021-08-28 00:09:54 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"ContainersReady\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Initialized\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:containerStatuses":{},"f:hostIP":{},"f:podIP":{},"f:podIPs":{".":{},"k:{\"ip\":\"10.244.3.89\"}":{".":{},"f:ip":{}}},"f:startTime":{}}}}]},Spec:PodSpec{Volumes:[]Volume{Volume{Name:default-token-d5hhc,VolumeSource:VolumeSource{HostPath:nil,EmptyDir:nil,GCEPersistentDisk:nil,AWSElasticBlockStore:nil,GitRepo:nil,Secret:&SecretVolumeSource{SecretName:default-token-d5hhc,Items:[]KeyToPath{},DefaultMode:*420,Optional:nil,},NFS:nil,ISCSI:nil,Glusterfs:nil,PersistentVolumeClaim:nil,RBD:nil,FlexVolume:nil,Cinder:nil,CephFS:nil,Flocker:nil,DownwardAPI:nil,FC:nil,AzureFile:nil,ConfigMap:nil,VsphereVolume:nil,Quobyte:nil,AzureDisk:nil,PhotonPersistentDisk:nil,PortworxVolume:nil,ScaleIO:nil,Projected:nil,StorageOS:nil,CSI:nil,Ephemeral:nil,},},},Containers:[]Container{Container{Name:httpd,Image:docker.io/library/httpd:2.4.38-alpine,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:default-token-d5hhc,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:nil,Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,},},RestartPolicy:Always,TerminationGracePeriodSeconds:*0,ActiveDeadlineSeconds:nil,DNSPolicy:ClusterFirst,NodeSelector:map[string]string{},ServiceAccountName:default,DeprecatedServiceAccount:default,NodeName:node1,HostNetwork:false,HostPID:false,HostIPC:false,SecurityContext:&PodSecurityContext{SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,SupplementalGroups:[],FSGroup:nil,RunAsGroup:nil,Sysctls:[]Sysctl{},WindowsOptions:nil,FSGroupChangePolicy:nil,SeccompProfile:nil,},ImagePullSecrets:[]LocalObjectReference{},Hostname:,Subdomain:,Affinity:nil,SchedulerName:default-scheduler,InitContainers:[]Container{},AutomountServiceAccountToken:nil,Tolerations:[]Toleration{Toleration{Key:node.kubernetes.io/not-ready,Operator:Exists,Value:,Effect:NoExecute,TolerationSeconds:*300,},Toleration{Key:node.kubernetes.io/unreachable,Operator:Exists,Value:,Effect:NoExecute,TolerationSeconds:*300,},},HostAliases:[]HostAlias{},PriorityClassName:,Priority:*0,DNSConfig:nil,ShareProcessNamespace:nil,ReadinessGates:[]PodReadinessGate{},RuntimeClassName:nil,EnableServiceLinks:*true,PreemptionPolicy:*PreemptLowerPriority,Overhead:ResourceList{},TopologySpreadConstraints:[]TopologySpreadConstraint{},EphemeralContainers:[]EphemeralContainer{},SetHostnameAsFQDN:nil,},Status:PodStatus{Phase:Pending,Conditions:[]PodCondition{PodCondition{Type:Initialized,Status:True,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:,Message:,},PodCondition{Type:Ready,Status:False,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:ContainersNotReady,Message:containers with unready status: [httpd],},PodCondition{Type:ContainersReady,Status:False,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:ContainersNotReady,Message:containers with unready status: [httpd],},PodCondition{Type:PodScheduled,Status:True,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:,Message:,},},Message:,Reason:,HostIP:10.10.190.207,PodIP:10.244.3.89,StartTime:2021-08-28 00:09:42 +0000 UTC,ContainerStatuses:[]ContainerStatus{ContainerStatus{Name:httpd,State:ContainerState{Waiting:&ContainerStateWaiting{Reason:ImagePullBackOff,Message:Back-off pulling image "docker.io/library/httpd:2.4.38-alpine",},Running:nil,Terminated:nil,},LastTerminationState:ContainerState{Waiting:nil,Running:nil,Terminated:nil,},Ready:false,RestartCount:0,Image:docker.io/library/httpd:2.4.38-alpine,ImageID:,ContainerID:,Started:*false,},},QOSClass:BestEffort,InitContainerStatuses:[]ContainerStatus{},NominatedNodeName:,PodIPs:[]PodIP{PodIP{IP:10.244.3.89,},},EphemeralContainerStatuses:[]ContainerStatus{},},} Aug 28 00:14:44.802: INFO: Pod "webserver-deployment-dd94f59b7-77lh5" is not available: &Pod{ObjectMeta:{webserver-deployment-dd94f59b7-77lh5 webserver-deployment-dd94f59b7- deployment-4063 /api/v1/namespaces/deployment-4063/pods/webserver-deployment-dd94f59b7-77lh5 a3043569-5a37-4d7d-8b33-4b743606a77b 74913 0 2021-08-28 00:09:42 +0000 UTC <nil> <nil> map[name:httpd pod-template-hash:dd94f59b7] map[k8s.v1.cni.cncf.io/network-status:[{ "name": "default-cni-network", "interface": "eth0", "ips": [ "10.244.4.145" ], "mac": "7e:44:11:fe:7c:2a", "default": true, "dns": {} }] k8s.v1.cni.cncf.io/networks-status:[{ "name": "default-cni-network", "interface": "eth0", "ips": [ "10.244.4.145" ], "mac": "7e:44:11:fe:7c:2a", "default": true, "dns": {} }] kubernetes.io/psp:collectd] [{apps/v1 ReplicaSet webserver-deployment-dd94f59b7 7163f4e0-9f27-4d87-a33d-f37e159ea03a 0xc0001a4d0f 0xc0001a4d70}] [] [{kube-controller-manager Update v1 2021-08-28 00:09:42 +0000 UTC FieldsV1 {"f:metadata":{"f:generateName":{},"f:labels":{".":{},"f:name":{},"f:pod-template-hash":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"7163f4e0-9f27-4d87-a33d-f37e159ea03a\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:containers":{"k:{\"name\":\"httpd\"}":{".":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:securityContext":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:terminationGracePeriodSeconds":{}}}} {kubelet Update v1 2021-08-28 00:10:00 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"ContainersReady\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Initialized\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:containerStatuses":{},"f:hostIP":{},"f:podIP":{},"f:podIPs":{".":{},"k:{\"ip\":\"10.244.4.145\"}":{".":{},"f:ip":{}}},"f:startTime":{}}}} {multus Update v1 2021-08-28 00:10:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:k8s.v1.cni.cncf.io/network-status":{},"f:k8s.v1.cni.cncf.io/networks-status":{}}}}}]},Spec:PodSpec{Volumes:[]Volume{Volume{Name:default-token-d5hhc,VolumeSource:VolumeSource{HostPath:nil,EmptyDir:nil,GCEPersistentDisk:nil,AWSElasticBlockStore:nil,GitRepo:nil,Secret:&SecretVolumeSource{SecretName:default-token-d5hhc,Items:[]KeyToPath{},DefaultMode:*420,Optional:nil,},NFS:nil,ISCSI:nil,Glusterfs:nil,PersistentVolumeClaim:nil,RBD:nil,FlexVolume:nil,Cinder:nil,CephFS:nil,Flocker:nil,DownwardAPI:nil,FC:nil,AzureFile:nil,ConfigMap:nil,VsphereVolume:nil,Quobyte:nil,AzureDisk:nil,PhotonPersistentDisk:nil,PortworxVolume:nil,ScaleIO:nil,Projected:nil,StorageOS:nil,CSI:nil,Ephemeral:nil,},},},Containers:[]Container{Container{Name:httpd,Image:docker.io/library/httpd:2.4.38-alpine,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:default-token-d5hhc,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:nil,Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,},},RestartPolicy:Always,TerminationGracePeriodSeconds:*0,ActiveDeadlineSeconds:nil,DNSPolicy:ClusterFirst,NodeSelector:map[string]string{},ServiceAccountName:default,DeprecatedServiceAccount:default,NodeName:node2,HostNetwork:false,HostPID:false,HostIPC:false,SecurityContext:&PodSecurityContext{SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,SupplementalGroups:[],FSGroup:nil,RunAsGroup:nil,Sysctls:[]Sysctl{},WindowsOptions:nil,FSGroupChangePolicy:nil,SeccompProfile:nil,},ImagePullSecrets:[]LocalObjectReference{},Hostname:,Subdomain:,Affinity:nil,SchedulerName:default-scheduler,InitContainers:[]Container{},AutomountServiceAccountToken:nil,Tolerations:[]Toleration{Toleration{Key:node.kubernetes.io/not-ready,Operator:Exists,Value:,Effect:NoExecute,TolerationSeconds:*300,},Toleration{Key:node.kubernetes.io/unreachable,Operator:Exists,Value:,Effect:NoExecute,TolerationSeconds:*300,},},HostAliases:[]HostAlias{},PriorityClassName:,Priority:*0,DNSConfig:nil,ShareProcessNamespace:nil,ReadinessGates:[]PodReadinessGate{},RuntimeClassName:nil,EnableServiceLinks:*true,PreemptionPolicy:*PreemptLowerPriority,Overhead:ResourceList{},TopologySpreadConstraints:[]TopologySpreadConstraint{},EphemeralContainers:[]EphemeralContainer{},SetHostnameAsFQDN:nil,},Status:PodStatus{Phase:Pending,Conditions:[]PodCondition{PodCondition{Type:Initialized,Status:True,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:,Message:,},PodCondition{Type:Ready,Status:False,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:ContainersNotReady,Message:containers with unready status: [httpd],},PodCondition{Type:ContainersReady,Status:False,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:ContainersNotReady,Message:containers with unready status: [httpd],},PodCondition{Type:PodScheduled,Status:True,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:,Message:,},},Message:,Reason:,HostIP:10.10.190.208,PodIP:10.244.4.145,StartTime:2021-08-28 00:09:42 +0000 UTC,ContainerStatuses:[]ContainerStatus{ContainerStatus{Name:httpd,State:ContainerState{Waiting:&ContainerStateWaiting{Reason:ImagePullBackOff,Message:Back-off pulling image "docker.io/library/httpd:2.4.38-alpine",},Running:nil,Terminated:nil,},LastTerminationState:ContainerState{Waiting:nil,Running:nil,Terminated:nil,},Ready:false,RestartCount:0,Image:docker.io/library/httpd:2.4.38-alpine,ImageID:,ContainerID:,Started:*false,},},QOSClass:BestEffort,InitContainerStatuses:[]ContainerStatus{},NominatedNodeName:,PodIPs:[]PodIP{PodIP{IP:10.244.4.145,},},EphemeralContainerStatuses:[]ContainerStatus{},},} Aug 28 00:14:44.802: INFO: Pod "webserver-deployment-dd94f59b7-bv7w9" is not available: &Pod{ObjectMeta:{webserver-deployment-dd94f59b7-bv7w9 webserver-deployment-dd94f59b7- deployment-4063 /api/v1/namespaces/deployment-4063/pods/webserver-deployment-dd94f59b7-bv7w9 5acd5ea8-9364-4481-9707-cf42db97204b 74599 0 2021-08-28 00:09:42 +0000 UTC <nil> <nil> map[name:httpd pod-template-hash:dd94f59b7] map[k8s.v1.cni.cncf.io/network-status:[{ "name": "default-cni-network", "interface": "eth0", "ips": [ "10.244.3.83" ], "mac": "36:b7:3c:25:83:24", "default": true, "dns": {} }] k8s.v1.cni.cncf.io/networks-status:[{ "name": "default-cni-network", "interface": "eth0", "ips": [ "10.244.3.83" ], "mac": "36:b7:3c:25:83:24", "default": true, "dns": {} }] kubernetes.io/psp:collectd] [{apps/v1 ReplicaSet webserver-deployment-dd94f59b7 7163f4e0-9f27-4d87-a33d-f37e159ea03a 0xc0001a5dff 0xc0001a5ea0}] [] [{kube-controller-manager Update v1 2021-08-28 00:09:42 +0000 UTC FieldsV1 {"f:metadata":{"f:generateName":{},"f:labels":{".":{},"f:name":{},"f:pod-template-hash":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"7163f4e0-9f27-4d87-a33d-f37e159ea03a\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:containers":{"k:{\"name\":\"httpd\"}":{".":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:securityContext":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:terminationGracePeriodSeconds":{}}}} {multus Update v1 2021-08-28 00:09:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:k8s.v1.cni.cncf.io/network-status":{},"f:k8s.v1.cni.cncf.io/networks-status":{}}}}} {kubelet Update v1 2021-08-28 00:09:48 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"ContainersReady\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Initialized\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:containerStatuses":{},"f:hostIP":{},"f:podIP":{},"f:podIPs":{".":{},"k:{\"ip\":\"10.244.3.83\"}":{".":{},"f:ip":{}}},"f:startTime":{}}}}]},Spec:PodSpec{Volumes:[]Volume{Volume{Name:default-token-d5hhc,VolumeSource:VolumeSource{HostPath:nil,EmptyDir:nil,GCEPersistentDisk:nil,AWSElasticBlockStore:nil,GitRepo:nil,Secret:&SecretVolumeSource{SecretName:default-token-d5hhc,Items:[]KeyToPath{},DefaultMode:*420,Optional:nil,},NFS:nil,ISCSI:nil,Glusterfs:nil,PersistentVolumeClaim:nil,RBD:nil,FlexVolume:nil,Cinder:nil,CephFS:nil,Flocker:nil,DownwardAPI:nil,FC:nil,AzureFile:nil,ConfigMap:nil,VsphereVolume:nil,Quobyte:nil,AzureDisk:nil,PhotonPersistentDisk:nil,PortworxVolume:nil,ScaleIO:nil,Projected:nil,StorageOS:nil,CSI:nil,Ephemeral:nil,},},},Containers:[]Container{Container{Name:httpd,Image:docker.io/library/httpd:2.4.38-alpine,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:default-token-d5hhc,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:nil,Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,},},RestartPolicy:Always,TerminationGracePeriodSeconds:*0,ActiveDeadlineSeconds:nil,DNSPolicy:ClusterFirst,NodeSelector:map[string]string{},ServiceAccountName:default,DeprecatedServiceAccount:default,NodeName:node1,HostNetwork:false,HostPID:false,HostIPC:false,SecurityContext:&PodSecurityContext{SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,SupplementalGroups:[],FSGroup:nil,RunAsGroup:nil,Sysctls:[]Sysctl{},WindowsOptions:nil,FSGroupChangePolicy:nil,SeccompProfile:nil,},ImagePullSecrets:[]LocalObjectReference{},Hostname:,Subdomain:,Affinity:nil,SchedulerName:default-scheduler,InitContainers:[]Container{},AutomountServiceAccountToken:nil,Tolerations:[]Toleration{Toleration{Key:node.kubernetes.io/not-ready,Operator:Exists,Value:,Effect:NoExecute,TolerationSeconds:*300,},Toleration{Key:node.kubernetes.io/unreachable,Operator:Exists,Value:,Effect:NoExecute,TolerationSeconds:*300,},},HostAliases:[]HostAlias{},PriorityClassName:,Priority:*0,DNSConfig:nil,ShareProcessNamespace:nil,ReadinessGates:[]PodReadinessGate{},RuntimeClassName:nil,EnableServiceLinks:*true,PreemptionPolicy:*PreemptLowerPriority,Overhead:ResourceList{},TopologySpreadConstraints:[]TopologySpreadConstraint{},EphemeralContainers:[]EphemeralContainer{},SetHostnameAsFQDN:nil,},Status:PodStatus{Phase:Pending,Conditions:[]PodCondition{PodCondition{Type:Initialized,Status:True,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:,Message:,},PodCondition{Type:Ready,Status:False,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:ContainersNotReady,Message:containers with unready status: [httpd],},PodCondition{Type:ContainersReady,Status:False,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:ContainersNotReady,Message:containers with unready status: [httpd],},PodCondition{Type:PodScheduled,Status:True,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:,Message:,},},Message:,Reason:,HostIP:10.10.190.207,PodIP:10.244.3.83,StartTime:2021-08-28 00:09:42 +0000 UTC,ContainerStatuses:[]ContainerStatus{ContainerStatus{Name:httpd,State:ContainerState{Waiting:&ContainerStateWaiting{Reason:ImagePullBackOff,Message:Back-off pulling image "docker.io/library/httpd:2.4.38-alpine",},Running:nil,Terminated:nil,},LastTerminationState:ContainerState{Waiting:nil,Running:nil,Terminated:nil,},Ready:false,RestartCount:0,Image:docker.io/library/httpd:2.4.38-alpine,ImageID:,ContainerID:,Started:*false,},},QOSClass:BestEffort,InitContainerStatuses:[]ContainerStatus{},NominatedNodeName:,PodIPs:[]PodIP{PodIP{IP:10.244.3.83,},},EphemeralContainerStatuses:[]ContainerStatus{},},} Aug 28 00:14:44.803: INFO: Pod "webserver-deployment-dd94f59b7-f4kxs" is not available: &Pod{ObjectMeta:{webserver-deployment-dd94f59b7-f4kxs webserver-deployment-dd94f59b7- deployment-4063 /api/v1/namespaces/deployment-4063/pods/webserver-deployment-dd94f59b7-f4kxs 77315150-228d-4fd6-b8fd-ea9aceec14bf 74884 0 2021-08-28 00:09:42 +0000 UTC <nil> <nil> map[name:httpd pod-template-hash:dd94f59b7] map[k8s.v1.cni.cncf.io/network-status:[{ "name": "default-cni-network", "interface": "eth0", "ips": [ "10.244.3.90" ], "mac": "06:1a:05:32:f8:b7", "default": true, "dns": {} }] k8s.v1.cni.cncf.io/networks-status:[{ "name": "default-cni-network", "interface": "eth0", "ips": [ "10.244.3.90" ], "mac": "06:1a:05:32:f8:b7", "default": true, "dns": {} }] kubernetes.io/psp:collectd] [{apps/v1 ReplicaSet webserver-deployment-dd94f59b7 7163f4e0-9f27-4d87-a33d-f37e159ea03a 0xc00248c2ef 0xc00248c300}] [] [{kube-controller-manager Update v1 2021-08-28 00:09:42 +0000 UTC FieldsV1 {"f:metadata":{"f:generateName":{},"f:labels":{".":{},"f:name":{},"f:pod-template-hash":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"7163f4e0-9f27-4d87-a33d-f37e159ea03a\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:containers":{"k:{\"name\":\"httpd\"}":{".":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:securityContext":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:terminationGracePeriodSeconds":{}}}} {kubelet Update v1 2021-08-28 00:09:54 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"ContainersReady\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Initialized\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:containerStatuses":{},"f:hostIP":{},"f:podIP":{},"f:podIPs":{".":{},"k:{\"ip\":\"10.244.3.90\"}":{".":{},"f:ip":{}}},"f:startTime":{}}}} {multus Update v1 2021-08-28 00:09:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:k8s.v1.cni.cncf.io/network-status":{},"f:k8s.v1.cni.cncf.io/networks-status":{}}}}}]},Spec:PodSpec{Volumes:[]Volume{Volume{Name:default-token-d5hhc,VolumeSource:VolumeSource{HostPath:nil,EmptyDir:nil,GCEPersistentDisk:nil,AWSElasticBlockStore:nil,GitRepo:nil,Secret:&SecretVolumeSource{SecretName:default-token-d5hhc,Items:[]KeyToPath{},DefaultMode:*420,Optional:nil,},NFS:nil,ISCSI:nil,Glusterfs:nil,PersistentVolumeClaim:nil,RBD:nil,FlexVolume:nil,Cinder:nil,CephFS:nil,Flocker:nil,DownwardAPI:nil,FC:nil,AzureFile:nil,ConfigMap:nil,VsphereVolume:nil,Quobyte:nil,AzureDisk:nil,PhotonPersistentDisk:nil,PortworxVolume:nil,ScaleIO:nil,Projected:nil,StorageOS:nil,CSI:nil,Ephemeral:nil,},},},Containers:[]Container{Container{Name:httpd,Image:docker.io/library/httpd:2.4.38-alpine,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:default-token-d5hhc,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:nil,Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,},},RestartPolicy:Always,TerminationGracePeriodSeconds:*0,ActiveDeadlineSeconds:nil,DNSPolicy:ClusterFirst,NodeSelector:map[string]string{},ServiceAccountName:default,DeprecatedServiceAccount:default,NodeName:node1,HostNetwork:false,HostPID:false,HostIPC:false,SecurityContext:&PodSecurityContext{SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,SupplementalGroups:[],FSGroup:nil,RunAsGroup:nil,Sysctls:[]Sysctl{},WindowsOptions:nil,FSGroupChangePolicy:nil,SeccompProfile:nil,},ImagePullSecrets:[]LocalObjectReference{},Hostname:,Subdomain:,Affinity:nil,SchedulerName:default-scheduler,InitContainers:[]Container{},AutomountServiceAccountToken:nil,Tolerations:[]Toleration{Toleration{Key:node.kubernetes.io/not-ready,Operator:Exists,Value:,Effect:NoExecute,TolerationSeconds:*300,},Toleration{Key:node.kubernetes.io/unreachable,Operator:Exists,Value:,Effect:NoExecute,TolerationSeconds:*300,},},HostAliases:[]HostAlias{},PriorityClassName:,Priority:*0,DNSConfig:nil,ShareProcessNamespace:nil,ReadinessGates:[]PodReadinessGate{},RuntimeClassName:nil,EnableServiceLinks:*true,PreemptionPolicy:*PreemptLowerPriority,Overhead:ResourceList{},TopologySpreadConstraints:[]TopologySpreadConstraint{},EphemeralContainers:[]EphemeralContainer{},SetHostnameAsFQDN:nil,},Status:PodStatus{Phase:Pending,Conditions:[]PodCondition{PodCondition{Type:Initialized,Status:True,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:,Message:,},PodCondition{Type:Ready,Status:False,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:ContainersNotReady,Message:containers with unready status: [httpd],},PodCondition{Type:ContainersReady,Status:False,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:ContainersNotReady,Message:containers with unready status: [httpd],},PodCondition{Type:PodScheduled,Status:True,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:,Message:,},},Message:,Reason:,HostIP:10.10.190.207,PodIP:10.244.3.90,StartTime:2021-08-28 00:09:42 +0000 UTC,ContainerStatuses:[]ContainerStatus{ContainerStatus{Name:httpd,State:ContainerState{Waiting:&ContainerStateWaiting{Reason:ImagePullBackOff,Message:Back-off pulling image "docker.io/library/httpd:2.4.38-alpine",},Running:nil,Terminated:nil,},LastTerminationState:ContainerState{Waiting:nil,Running:nil,Terminated:nil,},Ready:false,RestartCount:0,Image:docker.io/library/httpd:2.4.38-alpine,ImageID:,ContainerID:,Started:*false,},},QOSClass:BestEffort,InitContainerStatuses:[]ContainerStatus{},NominatedNodeName:,PodIPs:[]PodIP{PodIP{IP:10.244.3.90,},},EphemeralContainerStatuses:[]ContainerStatus{},},} Aug 28 00:14:44.803: INFO: Pod "webserver-deployment-dd94f59b7-hddhg" is not available: &Pod{ObjectMeta:{webserver-deployment-dd94f59b7-hddhg webserver-deployment-dd94f59b7- deployment-4063 /api/v1/namespaces/deployment-4063/pods/webserver-deployment-dd94f59b7-hddhg f93e3ebe-1cc6-4e8f-86e3-abd1f9b5699d 74820 0 2021-08-28 00:09:42 +0000 UTC <nil> <nil> map[name:httpd pod-template-hash:dd94f59b7] map[k8s.v1.cni.cncf.io/network-status:[{ "name": "default-cni-network", "interface": "eth0", "ips": [ "10.244.3.88" ], "mac": "7e:3c:58:44:17:46", "default": true, "dns": {} }] k8s.v1.cni.cncf.io/networks-status:[{ "name": "default-cni-network", "interface": "eth0", "ips": [ "10.244.3.88" ], "mac": "7e:3c:58:44:17:46", "default": true, "dns": {} }] kubernetes.io/psp:collectd] [{apps/v1 ReplicaSet webserver-deployment-dd94f59b7 7163f4e0-9f27-4d87-a33d-f37e159ea03a 0xc00248c4bf 0xc00248c4d0}] [] [{kube-controller-manager Update v1 2021-08-28 00:09:42 +0000 UTC FieldsV1 {"f:metadata":{"f:generateName":{},"f:labels":{".":{},"f:name":{},"f:pod-template-hash":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"7163f4e0-9f27-4d87-a33d-f37e159ea03a\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:containers":{"k:{\"name\":\"httpd\"}":{".":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:securityContext":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:terminationGracePeriodSeconds":{}}}} {kubelet Update v1 2021-08-28 00:09:51 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"ContainersReady\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Initialized\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:containerStatuses":{},"f:hostIP":{},"f:podIP":{},"f:podIPs":{".":{},"k:{\"ip\":\"10.244.3.88\"}":{".":{},"f:ip":{}}},"f:startTime":{}}}} {multus Update v1 2021-08-28 00:09:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:k8s.v1.cni.cncf.io/network-status":{},"f:k8s.v1.cni.cncf.io/networks-status":{}}}}}]},Spec:PodSpec{Volumes:[]Volume{Volume{Name:default-token-d5hhc,VolumeSource:VolumeSource{HostPath:nil,EmptyDir:nil,GCEPersistentDisk:nil,AWSElasticBlockStore:nil,GitRepo:nil,Secret:&SecretVolumeSource{SecretName:default-token-d5hhc,Items:[]KeyToPath{},DefaultMode:*420,Optional:nil,},NFS:nil,ISCSI:nil,Glusterfs:nil,PersistentVolumeClaim:nil,RBD:nil,FlexVolume:nil,Cinder:nil,CephFS:nil,Flocker:nil,DownwardAPI:nil,FC:nil,AzureFile:nil,ConfigMap:nil,VsphereVolume:nil,Quobyte:nil,AzureDisk:nil,PhotonPersistentDisk:nil,PortworxVolume:nil,ScaleIO:nil,Projected:nil,StorageOS:nil,CSI:nil,Ephemeral:nil,},},},Containers:[]Container{Container{Name:httpd,Image:docker.io/library/httpd:2.4.38-alpine,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:default-token-d5hhc,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:nil,Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,},},RestartPolicy:Always,TerminationGracePeriodSeconds:*0,ActiveDeadlineSeconds:nil,DNSPolicy:ClusterFirst,NodeSelector:map[string]string{},ServiceAccountName:default,DeprecatedServiceAccount:default,NodeName:node1,HostNetwork:false,HostPID:false,HostIPC:false,SecurityContext:&PodSecurityContext{SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,SupplementalGroups:[],FSGroup:nil,RunAsGroup:nil,Sysctls:[]Sysctl{},WindowsOptions:nil,FSGroupChangePolicy:nil,SeccompProfile:nil,},ImagePullSecrets:[]LocalObjectReference{},Hostname:,Subdomain:,Affinity:nil,SchedulerName:default-scheduler,InitContainers:[]Container{},AutomountServiceAccountToken:nil,Tolerations:[]Toleration{Toleration{Key:node.kubernetes.io/not-ready,Operator:Exists,Value:,Effect:NoExecute,TolerationSeconds:*300,},Toleration{Key:node.kubernetes.io/unreachable,Operator:Exists,Value:,Effect:NoExecute,TolerationSeconds:*300,},},HostAliases:[]HostAlias{},PriorityClassName:,Priority:*0,DNSConfig:nil,ShareProcessNamespace:nil,ReadinessGates:[]PodReadinessGate{},RuntimeClassName:nil,EnableServiceLinks:*true,PreemptionPolicy:*PreemptLowerPriority,Overhead:ResourceList{},TopologySpreadConstraints:[]TopologySpreadConstraint{},EphemeralContainers:[]EphemeralContainer{},SetHostnameAsFQDN:nil,},Status:PodStatus{Phase:Pending,Conditions:[]PodCondition{PodCondition{Type:Initialized,Status:True,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:,Message:,},PodCondition{Type:Ready,Status:False,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:ContainersNotReady,Message:containers with unready status: [httpd],},PodCondition{Type:ContainersReady,Status:False,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:ContainersNotReady,Message:containers with unready status: [httpd],},PodCondition{Type:PodScheduled,Status:True,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:,Message:,},},Message:,Reason:,HostIP:10.10.190.207,PodIP:10.244.3.88,StartTime:2021-08-28 00:09:42 +0000 UTC,ContainerStatuses:[]ContainerStatus{ContainerStatus{Name:httpd,State:ContainerState{Waiting:&ContainerStateWaiting{Reason:ImagePullBackOff,Message:Back-off pulling image "docker.io/library/httpd:2.4.38-alpine",},Running:nil,Terminated:nil,},LastTerminationState:ContainerState{Waiting:nil,Running:nil,Terminated:nil,},Ready:false,RestartCount:0,Image:docker.io/library/httpd:2.4.38-alpine,ImageID:,ContainerID:,Started:*false,},},QOSClass:BestEffort,InitContainerStatuses:[]ContainerStatus{},NominatedNodeName:,PodIPs:[]PodIP{PodIP{IP:10.244.3.88,},},EphemeralContainerStatuses:[]ContainerStatus{},},} Aug 28 00:14:44.803: INFO: Pod "webserver-deployment-dd94f59b7-ntnc5" is not available: &Pod{ObjectMeta:{webserver-deployment-dd94f59b7-ntnc5 webserver-deployment-dd94f59b7- deployment-4063 /api/v1/namespaces/deployment-4063/pods/webserver-deployment-dd94f59b7-ntnc5 dac2a93b-fafa-4fd2-ba70-120100e0be01 75034 0 2021-08-28 00:09:42 +0000 UTC <nil> <nil> map[name:httpd pod-template-hash:dd94f59b7] map[k8s.v1.cni.cncf.io/network-status:[{ "name": "default-cni-network", "interface": "eth0", "ips": [ "10.244.4.141" ], "mac": "da:08:bf:b4:0c:28", "default": true, "dns": {} }] k8s.v1.cni.cncf.io/networks-status:[{ "name": "default-cni-network", "interface": "eth0", "ips": [ "10.244.4.141" ], "mac": "da:08:bf:b4:0c:28", "default": true, "dns": {} }] kubernetes.io/psp:collectd] [{apps/v1 ReplicaSet webserver-deployment-dd94f59b7 7163f4e0-9f27-4d87-a33d-f37e159ea03a 0xc00248c68f 0xc00248c6a0}] [] [{kube-controller-manager Update v1 2021-08-28 00:09:42 +0000 UTC FieldsV1 {"f:metadata":{"f:generateName":{},"f:labels":{".":{},"f:name":{},"f:pod-template-hash":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"7163f4e0-9f27-4d87-a33d-f37e159ea03a\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:containers":{"k:{\"name\":\"httpd\"}":{".":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:securityContext":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:terminationGracePeriodSeconds":{}}}} {multus Update v1 2021-08-28 00:09:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:k8s.v1.cni.cncf.io/network-status":{},"f:k8s.v1.cni.cncf.io/networks-status":{}}}}} {kubelet Update v1 2021-08-28 00:09:55 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"ContainersReady\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Initialized\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:containerStatuses":{},"f:hostIP":{},"f:podIP":{},"f:podIPs":{".":{},"k:{\"ip\":\"10.244.4.141\"}":{".":{},"f:ip":{}}},"f:startTime":{}}}}]},Spec:PodSpec{Volumes:[]Volume{Volume{Name:default-token-d5hhc,VolumeSource:VolumeSource{HostPath:nil,EmptyDir:nil,GCEPersistentDisk:nil,AWSElasticBlockStore:nil,GitRepo:nil,Secret:&SecretVolumeSource{SecretName:default-token-d5hhc,Items:[]KeyToPath{},DefaultMode:*420,Optional:nil,},NFS:nil,ISCSI:nil,Glusterfs:nil,PersistentVolumeClaim:nil,RBD:nil,FlexVolume:nil,Cinder:nil,CephFS:nil,Flocker:nil,DownwardAPI:nil,FC:nil,AzureFile:nil,ConfigMap:nil,VsphereVolume:nil,Quobyte:nil,AzureDisk:nil,PhotonPersistentDisk:nil,PortworxVolume:nil,ScaleIO:nil,Projected:nil,StorageOS:nil,CSI:nil,Ephemeral:nil,},},},Containers:[]Container{Container{Name:httpd,Image:docker.io/library/httpd:2.4.38-alpine,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:default-token-d5hhc,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:nil,Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,},},RestartPolicy:Always,TerminationGracePeriodSeconds:*0,ActiveDeadlineSeconds:nil,DNSPolicy:ClusterFirst,NodeSelector:map[string]string{},ServiceAccountName:default,DeprecatedServiceAccount:default,NodeName:node2,HostNetwork:false,HostPID:false,HostIPC:false,SecurityContext:&PodSecurityContext{SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,SupplementalGroups:[],FSGroup:nil,RunAsGroup:nil,Sysctls:[]Sysctl{},WindowsOptions:nil,FSGroupChangePolicy:nil,SeccompProfile:nil,},ImagePullSecrets:[]LocalObjectReference{},Hostname:,Subdomain:,Affinity:nil,SchedulerName:default-scheduler,InitContainers:[]Container{},AutomountServiceAccountToken:nil,Tolerations:[]Toleration{Toleration{Key:node.kubernetes.io/not-ready,Operator:Exists,Value:,Effect:NoExecute,TolerationSeconds:*300,},Toleration{Key:node.kubernetes.io/unreachable,Operator:Exists,Value:,Effect:NoExecute,TolerationSeconds:*300,},},HostAliases:[]HostAlias{},PriorityClassName:,Priority:*0,DNSConfig:nil,ShareProcessNamespace:nil,ReadinessGates:[]PodReadinessGate{},RuntimeClassName:nil,EnableServiceLinks:*true,PreemptionPolicy:*PreemptLowerPriority,Overhead:ResourceList{},TopologySpreadConstraints:[]TopologySpreadConstraint{},EphemeralContainers:[]EphemeralContainer{},SetHostnameAsFQDN:nil,},Status:PodStatus{Phase:Pending,Conditions:[]PodCondition{PodCondition{Type:Initialized,Status:True,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:,Message:,},PodCondition{Type:Ready,Status:False,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:ContainersNotReady,Message:containers with unready status: [httpd],},PodCondition{Type:ContainersReady,Status:False,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:ContainersNotReady,Message:containers with unready status: [httpd],},PodCondition{Type:PodScheduled,Status:True,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:,Message:,},},Message:,Reason:,HostIP:10.10.190.208,PodIP:10.244.4.141,StartTime:2021-08-28 00:09:42 +0000 UTC,ContainerStatuses:[]ContainerStatus{ContainerStatus{Name:httpd,State:ContainerState{Waiting:&ContainerStateWaiting{Reason:ImagePullBackOff,Message:Back-off pulling image "docker.io/library/httpd:2.4.38-alpine",},Running:nil,Terminated:nil,},LastTerminationState:ContainerState{Waiting:nil,Running:nil,Terminated:nil,},Ready:false,RestartCount:0,Image:docker.io/library/httpd:2.4.38-alpine,ImageID:,ContainerID:,Started:*false,},},QOSClass:BestEffort,InitContainerStatuses:[]ContainerStatus{},NominatedNodeName:,PodIPs:[]PodIP{PodIP{IP:10.244.4.141,},},EphemeralContainerStatuses:[]ContainerStatus{},},} Aug 28 00:14:44.804: INFO: Pod "webserver-deployment-dd94f59b7-rl54s" is not available: &Pod{ObjectMeta:{webserver-deployment-dd94f59b7-rl54s webserver-deployment-dd94f59b7- deployment-4063 /api/v1/namespaces/deployment-4063/pods/webserver-deployment-dd94f59b7-rl54s c7161c61-c169-4826-a3df-12653f394209 75020 0 2021-08-28 00:09:42 +0000 UTC <nil> <nil> map[name:httpd pod-template-hash:dd94f59b7] map[k8s.v1.cni.cncf.io/network-status:[{ "name": "default-cni-network", "interface": "eth0", "ips": [ "10.244.4.140" ], "mac": "8e:1b:54:04:19:51", "default": true, "dns": {} }] k8s.v1.cni.cncf.io/networks-status:[{ "name": "default-cni-network", "interface": "eth0", "ips": [ "10.244.4.140" ], "mac": "8e:1b:54:04:19:51", "default": true, "dns": {} }] kubernetes.io/psp:collectd] [{apps/v1 ReplicaSet webserver-deployment-dd94f59b7 7163f4e0-9f27-4d87-a33d-f37e159ea03a 0xc00248c85f 0xc00248c870}] [] [{kube-controller-manager Update v1 2021-08-28 00:09:42 +0000 UTC FieldsV1 {"f:metadata":{"f:generateName":{},"f:labels":{".":{},"f:name":{},"f:pod-template-hash":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"7163f4e0-9f27-4d87-a33d-f37e159ea03a\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:containers":{"k:{\"name\":\"httpd\"}":{".":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:securityContext":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:terminationGracePeriodSeconds":{}}}} {kubelet Update v1 2021-08-28 00:09:52 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"ContainersReady\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Initialized\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:containerStatuses":{},"f:hostIP":{},"f:podIP":{},"f:podIPs":{".":{},"k:{\"ip\":\"10.244.4.140\"}":{".":{},"f:ip":{}}},"f:startTime":{}}}} {multus Update v1 2021-08-28 00:09:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:k8s.v1.cni.cncf.io/network-status":{},"f:k8s.v1.cni.cncf.io/networks-status":{}}}}}]},Spec:PodSpec{Volumes:[]Volume{Volume{Name:default-token-d5hhc,VolumeSource:VolumeSource{HostPath:nil,EmptyDir:nil,GCEPersistentDisk:nil,AWSElasticBlockStore:nil,GitRepo:nil,Secret:&SecretVolumeSource{SecretName:default-token-d5hhc,Items:[]KeyToPath{},DefaultMode:*420,Optional:nil,},NFS:nil,ISCSI:nil,Glusterfs:nil,PersistentVolumeClaim:nil,RBD:nil,FlexVolume:nil,Cinder:nil,CephFS:nil,Flocker:nil,DownwardAPI:nil,FC:nil,AzureFile:nil,ConfigMap:nil,VsphereVolume:nil,Quobyte:nil,AzureDisk:nil,PhotonPersistentDisk:nil,PortworxVolume:nil,ScaleIO:nil,Projected:nil,StorageOS:nil,CSI:nil,Ephemeral:nil,},},},Containers:[]Container{Container{Name:httpd,Image:docker.io/library/httpd:2.4.38-alpine,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:default-token-d5hhc,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:nil,Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,},},RestartPolicy:Always,TerminationGracePeriodSeconds:*0,ActiveDeadlineSeconds:nil,DNSPolicy:ClusterFirst,NodeSelector:map[string]string{},ServiceAccountName:default,DeprecatedServiceAccount:default,NodeName:node2,HostNetwork:false,HostPID:false,HostIPC:false,SecurityContext:&PodSecurityContext{SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,SupplementalGroups:[],FSGroup:nil,RunAsGroup:nil,Sysctls:[]Sysctl{},WindowsOptions:nil,FSGroupChangePolicy:nil,SeccompProfile:nil,},ImagePullSecrets:[]LocalObjectReference{},Hostname:,Subdomain:,Affinity:nil,SchedulerName:default-scheduler,InitContainers:[]Container{},AutomountServiceAccountToken:nil,Tolerations:[]Toleration{Toleration{Key:node.kubernetes.io/not-ready,Operator:Exists,Value:,Effect:NoExecute,TolerationSeconds:*300,},Toleration{Key:node.kubernetes.io/unreachable,Operator:Exists,Value:,Effect:NoExecute,TolerationSeconds:*300,},},HostAliases:[]HostAlias{},PriorityClassName:,Priority:*0,DNSConfig:nil,ShareProcessNamespace:nil,ReadinessGates:[]PodReadinessGate{},RuntimeClassName:nil,EnableServiceLinks:*true,PreemptionPolicy:*PreemptLowerPriority,Overhead:ResourceList{},TopologySpreadConstraints:[]TopologySpreadConstraint{},EphemeralContainers:[]EphemeralContainer{},SetHostnameAsFQDN:nil,},Status:PodStatus{Phase:Pending,Conditions:[]PodCondition{PodCondition{Type:Initialized,Status:True,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:,Message:,},PodCondition{Type:Ready,Status:False,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:ContainersNotReady,Message:containers with unready status: [httpd],},PodCondition{Type:ContainersReady,Status:False,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:ContainersNotReady,Message:containers with unready status: [httpd],},PodCondition{Type:PodScheduled,Status:True,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:,Message:,},},Message:,Reason:,HostIP:10.10.190.208,PodIP:10.244.4.140,StartTime:2021-08-28 00:09:42 +0000 UTC,ContainerStatuses:[]ContainerStatus{ContainerStatus{Name:httpd,State:ContainerState{Waiting:&ContainerStateWaiting{Reason:ImagePullBackOff,Message:Back-off pulling image "docker.io/library/httpd:2.4.38-alpine",},Running:nil,Terminated:nil,},LastTerminationState:ContainerState{Waiting:nil,Running:nil,Terminated:nil,},Ready:false,RestartCount:0,Image:docker.io/library/httpd:2.4.38-alpine,ImageID:,ContainerID:,Started:*false,},},QOSClass:BestEffort,InitContainerStatuses:[]ContainerStatus{},NominatedNodeName:,PodIPs:[]PodIP{PodIP{IP:10.244.4.140,},},EphemeralContainerStatuses:[]ContainerStatus{},},} Aug 28 00:14:44.804: INFO: Pod "webserver-deployment-dd94f59b7-wnjgv" is not available: &Pod{ObjectMeta:{webserver-deployment-dd94f59b7-wnjgv webserver-deployment-dd94f59b7- deployment-4063 /api/v1/namespaces/deployment-4063/pods/webserver-deployment-dd94f59b7-wnjgv b4d6c573-bea8-4047-ab85-46c345755905 74772 0 2021-08-28 00:09:42 +0000 UTC <nil> <nil> map[name:httpd pod-template-hash:dd94f59b7] map[k8s.v1.cni.cncf.io/network-status:[{ "name": "default-cni-network", "interface": "eth0", "ips": [ "10.244.3.92" ], "mac": "56:78:44:72:56:c7", "default": true, "dns": {} }] k8s.v1.cni.cncf.io/networks-status:[{ "name": "default-cni-network", "interface": "eth0", "ips": [ "10.244.3.92" ], "mac": "56:78:44:72:56:c7", "default": true, "dns": {} }] kubernetes.io/psp:collectd] [{apps/v1 ReplicaSet webserver-deployment-dd94f59b7 7163f4e0-9f27-4d87-a33d-f37e159ea03a 0xc00248ca2f 0xc00248ca40}] [] [{kube-controller-manager Update v1 2021-08-28 00:09:42 +0000 UTC FieldsV1 {"f:metadata":{"f:generateName":{},"f:labels":{".":{},"f:name":{},"f:pod-template-hash":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"7163f4e0-9f27-4d87-a33d-f37e159ea03a\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:containers":{"k:{\"name\":\"httpd\"}":{".":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:securityContext":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:terminationGracePeriodSeconds":{}}}} {kubelet Update v1 2021-08-28 00:09:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"ContainersReady\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Initialized\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:containerStatuses":{},"f:hostIP":{},"f:podIP":{},"f:podIPs":{".":{},"k:{\"ip\":\"10.244.3.92\"}":{".":{},"f:ip":{}}},"f:startTime":{}}}} {multus Update v1 2021-08-28 00:09:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:k8s.v1.cni.cncf.io/network-status":{},"f:k8s.v1.cni.cncf.io/networks-status":{}}}}}]},Spec:PodSpec{Volumes:[]Volume{Volume{Name:default-token-d5hhc,VolumeSource:VolumeSource{HostPath:nil,EmptyDir:nil,GCEPersistentDisk:nil,AWSElasticBlockStore:nil,GitRepo:nil,Secret:&SecretVolumeSource{SecretName:default-token-d5hhc,Items:[]KeyToPath{},DefaultMode:*420,Optional:nil,},NFS:nil,ISCSI:nil,Glusterfs:nil,PersistentVolumeClaim:nil,RBD:nil,FlexVolume:nil,Cinder:nil,CephFS:nil,Flocker:nil,DownwardAPI:nil,FC:nil,AzureFile:nil,ConfigMap:nil,VsphereVolume:nil,Quobyte:nil,AzureDisk:nil,PhotonPersistentDisk:nil,PortworxVolume:nil,ScaleIO:nil,Projected:nil,StorageOS:nil,CSI:nil,Ephemeral:nil,},},},Containers:[]Container{Container{Name:httpd,Image:docker.io/library/httpd:2.4.38-alpine,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:default-token-d5hhc,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:nil,Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,},},RestartPolicy:Always,TerminationGracePeriodSeconds:*0,ActiveDeadlineSeconds:nil,DNSPolicy:ClusterFirst,NodeSelector:map[string]string{},ServiceAccountName:default,DeprecatedServiceAccount:default,NodeName:node1,HostNetwork:false,HostPID:false,HostIPC:false,SecurityContext:&PodSecurityContext{SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,SupplementalGroups:[],FSGroup:nil,RunAsGroup:nil,Sysctls:[]Sysctl{},WindowsOptions:nil,FSGroupChangePolicy:nil,SeccompProfile:nil,},ImagePullSecrets:[]LocalObjectReference{},Hostname:,Subdomain:,Affinity:nil,SchedulerName:default-scheduler,InitContainers:[]Container{},AutomountServiceAccountToken:nil,Tolerations:[]Toleration{Toleration{Key:node.kubernetes.io/not-ready,Operator:Exists,Value:,Effect:NoExecute,TolerationSeconds:*300,},Toleration{Key:node.kubernetes.io/unreachable,Operator:Exists,Value:,Effect:NoExecute,TolerationSeconds:*300,},},HostAliases:[]HostAlias{},PriorityClassName:,Priority:*0,DNSConfig:nil,ShareProcessNamespace:nil,ReadinessGates:[]PodReadinessGate{},RuntimeClassName:nil,EnableServiceLinks:*true,PreemptionPolicy:*PreemptLowerPriority,Overhead:ResourceList{},TopologySpreadConstraints:[]TopologySpreadConstraint{},EphemeralContainers:[]EphemeralContainer{},SetHostnameAsFQDN:nil,},Status:PodStatus{Phase:Pending,Conditions:[]PodCondition{PodCondition{Type:Initialized,Status:True,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:,Message:,},PodCondition{Type:Ready,Status:False,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:ContainersNotReady,Message:containers with unready status: [httpd],},PodCondition{Type:ContainersReady,Status:False,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:ContainersNotReady,Message:containers with unready status: [httpd],},PodCondition{Type:PodScheduled,Status:True,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-08-28 00:09:42 +0000 UTC,Reason:,Message:,},},Message:,Reason:,HostIP:10.10.190.207,PodIP:10.244.3.92,StartTime:2021-08-28 00:09:42 +0000 UTC,ContainerStatuses:[]ContainerStatus{ContainerStatus{Name:httpd,State:ContainerState{Waiting:&ContainerStateWaiting{Reason:ImagePullBackOff,Message:Back-off pulling image "docker.io/library/httpd:2.4.38-alpine",},Running:nil,Terminated:nil,},LastTerminationState:ContainerState{Waiting:nil,Running:nil,Terminated:nil,},Ready:false,RestartCount:0,Image:docker.io/library/httpd:2.4.38-alpine,ImageID:,ContainerID:,Started:*false,},},QOSClass:BestEffort,InitContainerStatuses:[]ContainerStatus{},NominatedNodeName:,PodIPs:[]PodIP{PodIP{IP:10.244.3.92,},},EphemeralContainerStatuses:[]ContainerStatus{},},} [AfterEach] [sig-apps] Deployment /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "deployment-4063". STEP: Found 101 events. Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:42 +0000 UTC - event for webserver-deployment: {deployment-controller } ScalingReplicaSet: Scaled up replica set webserver-deployment-dd94f59b7 to 10 Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:42 +0000 UTC - event for webserver-deployment-dd94f59b7: {replicaset-controller } SuccessfulCreate: Created pod: webserver-deployment-dd94f59b7-bv7w9 Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:42 +0000 UTC - event for webserver-deployment-dd94f59b7: {replicaset-controller } SuccessfulCreate: Created pod: webserver-deployment-dd94f59b7-2z9dr Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:42 +0000 UTC - event for webserver-deployment-dd94f59b7: {replicaset-controller } SuccessfulCreate: (combined from similar events): Created pod: webserver-deployment-dd94f59b7-ntnc5 Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:42 +0000 UTC - event for webserver-deployment-dd94f59b7: {replicaset-controller } SuccessfulCreate: Created pod: webserver-deployment-dd94f59b7-f4kxs Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:42 +0000 UTC - event for webserver-deployment-dd94f59b7: {replicaset-controller } SuccessfulCreate: Created pod: webserver-deployment-dd94f59b7-56q7x Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:42 +0000 UTC - event for webserver-deployment-dd94f59b7: {replicaset-controller } SuccessfulCreate: Created pod: webserver-deployment-dd94f59b7-4jn4k Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:42 +0000 UTC - event for webserver-deployment-dd94f59b7: {replicaset-controller } SuccessfulCreate: Created pod: webserver-deployment-dd94f59b7-wnjgv Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:42 +0000 UTC - event for webserver-deployment-dd94f59b7: {replicaset-controller } SuccessfulCreate: Created pod: webserver-deployment-dd94f59b7-rl54s Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:42 +0000 UTC - event for webserver-deployment-dd94f59b7: {replicaset-controller } SuccessfulCreate: Created pod: webserver-deployment-dd94f59b7-77lh5 Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:42 +0000 UTC - event for webserver-deployment-dd94f59b7: {replicaset-controller } SuccessfulCreate: Created pod: webserver-deployment-dd94f59b7-hddhg Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:42 +0000 UTC - event for webserver-deployment-dd94f59b7-2z9dr: {default-scheduler } Scheduled: Successfully assigned deployment-4063/webserver-deployment-dd94f59b7-2z9dr to node2 Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:42 +0000 UTC - event for webserver-deployment-dd94f59b7-4jn4k: {default-scheduler } Scheduled: Successfully assigned deployment-4063/webserver-deployment-dd94f59b7-4jn4k to node2 Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:42 +0000 UTC - event for webserver-deployment-dd94f59b7-56q7x: {default-scheduler } Scheduled: Successfully assigned deployment-4063/webserver-deployment-dd94f59b7-56q7x to node1 Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:42 +0000 UTC - event for webserver-deployment-dd94f59b7-77lh5: {default-scheduler } Scheduled: Successfully assigned deployment-4063/webserver-deployment-dd94f59b7-77lh5 to node2 Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:42 +0000 UTC - event for webserver-deployment-dd94f59b7-bv7w9: {default-scheduler } Scheduled: Successfully assigned deployment-4063/webserver-deployment-dd94f59b7-bv7w9 to node1 Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:42 +0000 UTC - event for webserver-deployment-dd94f59b7-f4kxs: {default-scheduler } Scheduled: Successfully assigned deployment-4063/webserver-deployment-dd94f59b7-f4kxs to node1 Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:42 +0000 UTC - event for webserver-deployment-dd94f59b7-hddhg: {default-scheduler } Scheduled: Successfully assigned deployment-4063/webserver-deployment-dd94f59b7-hddhg to node1 Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:42 +0000 UTC - event for webserver-deployment-dd94f59b7-ntnc5: {default-scheduler } Scheduled: Successfully assigned deployment-4063/webserver-deployment-dd94f59b7-ntnc5 to node2 Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:42 +0000 UTC - event for webserver-deployment-dd94f59b7-rl54s: {default-scheduler } Scheduled: Successfully assigned deployment-4063/webserver-deployment-dd94f59b7-rl54s to node2 Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:42 +0000 UTC - event for webserver-deployment-dd94f59b7-wnjgv: {default-scheduler } Scheduled: Successfully assigned deployment-4063/webserver-deployment-dd94f59b7-wnjgv to node1 Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:45 +0000 UTC - event for webserver-deployment-dd94f59b7-bv7w9: {kubelet node1} Pulling: Pulling image "docker.io/library/httpd:2.4.38-alpine" Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:45 +0000 UTC - event for webserver-deployment-dd94f59b7-bv7w9: {multus } AddedInterface: Add eth0 [10.244.3.83/24] Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:46 +0000 UTC - event for webserver-deployment-dd94f59b7-bv7w9: {kubelet node1} Failed: Error: ErrImagePull Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:46 +0000 UTC - event for webserver-deployment-dd94f59b7-bv7w9: {kubelet node1} Failed: Failed to pull image "docker.io/library/httpd:2.4.38-alpine": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:46 +0000 UTC - event for webserver-deployment-dd94f59b7-f4kxs: {multus } AddedInterface: Add eth0 [10.244.3.84/24] Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:46 +0000 UTC - event for webserver-deployment-dd94f59b7-hddhg: {multus } AddedInterface: Add eth0 [10.244.3.86/24] Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:47 +0000 UTC - event for webserver-deployment-dd94f59b7-56q7x: {multus } AddedInterface: Add eth0 [10.244.3.85/24] Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:47 +0000 UTC - event for webserver-deployment-dd94f59b7-56q7x: {kubelet node1} Pulling: Pulling image "docker.io/library/httpd:2.4.38-alpine" Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:47 +0000 UTC - event for webserver-deployment-dd94f59b7-f4kxs: {kubelet node1} Pulling: Pulling image "docker.io/library/httpd:2.4.38-alpine" Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:47 +0000 UTC - event for webserver-deployment-dd94f59b7-hddhg: {kubelet node1} Pulling: Pulling image "docker.io/library/httpd:2.4.38-alpine" Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:47 +0000 UTC - event for webserver-deployment-dd94f59b7-wnjgv: {kubelet node1} Pulling: Pulling image "docker.io/library/httpd:2.4.38-alpine" Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:47 +0000 UTC - event for webserver-deployment-dd94f59b7-wnjgv: {multus } AddedInterface: Add eth0 [10.244.3.87/24] Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:48 +0000 UTC - event for webserver-deployment-dd94f59b7-2z9dr: {multus } AddedInterface: Add eth0 [10.244.4.136/24] Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:48 +0000 UTC - event for webserver-deployment-dd94f59b7-2z9dr: {kubelet node2} Pulling: Pulling image "docker.io/library/httpd:2.4.38-alpine" Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:48 +0000 UTC - event for webserver-deployment-dd94f59b7-4jn4k: {kubelet node2} Pulling: Pulling image "docker.io/library/httpd:2.4.38-alpine" Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:48 +0000 UTC - event for webserver-deployment-dd94f59b7-4jn4k: {multus } AddedInterface: Add eth0 [10.244.4.137/24] Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:48 +0000 UTC - event for webserver-deployment-dd94f59b7-77lh5: {kubelet node2} Pulling: Pulling image "docker.io/library/httpd:2.4.38-alpine" Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:48 +0000 UTC - event for webserver-deployment-dd94f59b7-77lh5: {multus } AddedInterface: Add eth0 [10.244.4.139/24] Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:48 +0000 UTC - event for webserver-deployment-dd94f59b7-bv7w9: {kubelet node1} Failed: Error: ImagePullBackOff Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:48 +0000 UTC - event for webserver-deployment-dd94f59b7-bv7w9: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/httpd:2.4.38-alpine" Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:48 +0000 UTC - event for webserver-deployment-dd94f59b7-hddhg: {kubelet node1} Failed: Error: ErrImagePull Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:48 +0000 UTC - event for webserver-deployment-dd94f59b7-hddhg: {kubelet node1} Failed: Failed to pull image "docker.io/library/httpd:2.4.38-alpine": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:48 +0000 UTC - event for webserver-deployment-dd94f59b7-ntnc5: {multus } AddedInterface: Add eth0 [10.244.4.138/24] Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:48 +0000 UTC - event for webserver-deployment-dd94f59b7-ntnc5: {kubelet node2} Pulling: Pulling image "docker.io/library/httpd:2.4.38-alpine" Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:48 +0000 UTC - event for webserver-deployment-dd94f59b7-rl54s: {kubelet node2} Pulling: Pulling image "docker.io/library/httpd:2.4.38-alpine" Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:48 +0000 UTC - event for webserver-deployment-dd94f59b7-rl54s: {multus } AddedInterface: Add eth0 [10.244.4.135/24] Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:49 +0000 UTC - event for webserver-deployment-dd94f59b7-hddhg: {kubelet node1} SandboxChanged: Pod sandbox changed, it will be killed and re-created. Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:49 +0000 UTC - event for webserver-deployment-dd94f59b7-rl54s: {kubelet node2} Failed: Error: ErrImagePull Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:49 +0000 UTC - event for webserver-deployment-dd94f59b7-rl54s: {kubelet node2} Failed: Failed to pull image "docker.io/library/httpd:2.4.38-alpine": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:50 +0000 UTC - event for webserver-deployment-dd94f59b7-56q7x: {kubelet node1} Failed: Failed to pull image "docker.io/library/httpd:2.4.38-alpine": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:50 +0000 UTC - event for webserver-deployment-dd94f59b7-56q7x: {kubelet node1} SandboxChanged: Pod sandbox changed, it will be killed and re-created. Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:50 +0000 UTC - event for webserver-deployment-dd94f59b7-56q7x: {kubelet node1} Failed: Error: ErrImagePull Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:50 +0000 UTC - event for webserver-deployment-dd94f59b7-f4kxs: {kubelet node1} Failed: Failed to pull image "docker.io/library/httpd:2.4.38-alpine": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:50 +0000 UTC - event for webserver-deployment-dd94f59b7-f4kxs: {kubelet node1} Failed: Error: ErrImagePull Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:50 +0000 UTC - event for webserver-deployment-dd94f59b7-ntnc5: {kubelet node2} Failed: Error: ErrImagePull Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:50 +0000 UTC - event for webserver-deployment-dd94f59b7-ntnc5: {kubelet node2} Failed: Failed to pull image "docker.io/library/httpd:2.4.38-alpine": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:50 +0000 UTC - event for webserver-deployment-dd94f59b7-rl54s: {kubelet node2} SandboxChanged: Pod sandbox changed, it will be killed and re-created. Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:51 +0000 UTC - event for webserver-deployment-dd94f59b7-4jn4k: {kubelet node2} Failed: Error: ErrImagePull Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:51 +0000 UTC - event for webserver-deployment-dd94f59b7-4jn4k: {kubelet node2} Failed: Failed to pull image "docker.io/library/httpd:2.4.38-alpine": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:51 +0000 UTC - event for webserver-deployment-dd94f59b7-f4kxs: {kubelet node1} SandboxChanged: Pod sandbox changed, it will be killed and re-created. Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:51 +0000 UTC - event for webserver-deployment-dd94f59b7-hddhg: {multus } AddedInterface: Add eth0 [10.244.3.88/24] Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:51 +0000 UTC - event for webserver-deployment-dd94f59b7-hddhg: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/httpd:2.4.38-alpine" Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:51 +0000 UTC - event for webserver-deployment-dd94f59b7-hddhg: {kubelet node1} Failed: Error: ImagePullBackOff Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:51 +0000 UTC - event for webserver-deployment-dd94f59b7-ntnc5: {kubelet node2} SandboxChanged: Pod sandbox changed, it will be killed and re-created. Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:51 +0000 UTC - event for webserver-deployment-dd94f59b7-wnjgv: {kubelet node1} Failed: Error: ErrImagePull Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:51 +0000 UTC - event for webserver-deployment-dd94f59b7-wnjgv: {kubelet node1} Failed: Failed to pull image "docker.io/library/httpd:2.4.38-alpine": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:52 +0000 UTC - event for webserver-deployment-dd94f59b7-2z9dr: {kubelet node2} Failed: Failed to pull image "docker.io/library/httpd:2.4.38-alpine": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:52 +0000 UTC - event for webserver-deployment-dd94f59b7-2z9dr: {kubelet node2} Failed: Error: ErrImagePull Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:52 +0000 UTC - event for webserver-deployment-dd94f59b7-4jn4k: {kubelet node2} SandboxChanged: Pod sandbox changed, it will be killed and re-created. Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:52 +0000 UTC - event for webserver-deployment-dd94f59b7-rl54s: {kubelet node2} BackOff: Back-off pulling image "docker.io/library/httpd:2.4.38-alpine" Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:52 +0000 UTC - event for webserver-deployment-dd94f59b7-rl54s: {kubelet node2} Failed: Error: ImagePullBackOff Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:52 +0000 UTC - event for webserver-deployment-dd94f59b7-rl54s: {multus } AddedInterface: Add eth0 [10.244.4.140/24] Aug 28 00:14:44.809: INFO: At 2021-08-28 00:09:53 +0000 UTC - event for webserver-deployment-dd94f59b7-56q7x: {kubelet node1} Failed: Error: ImagePullBackOff Aug 28 00:14:44.810: INFO: At 2021-08-28 00:09:53 +0000 UTC - event for webserver-deployment-dd94f59b7-56q7x: {multus } AddedInterface: Add eth0 [10.244.3.89/24] Aug 28 00:14:44.810: INFO: At 2021-08-28 00:09:53 +0000 UTC - event for webserver-deployment-dd94f59b7-56q7x: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/httpd:2.4.38-alpine" Aug 28 00:14:44.810: INFO: At 2021-08-28 00:09:53 +0000 UTC - event for webserver-deployment-dd94f59b7-77lh5: {kubelet node2} Failed: Error: ErrImagePull Aug 28 00:14:44.810: INFO: At 2021-08-28 00:09:53 +0000 UTC - event for webserver-deployment-dd94f59b7-77lh5: {kubelet node2} Failed: Failed to pull image "docker.io/library/httpd:2.4.38-alpine": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit Aug 28 00:14:44.810: INFO: At 2021-08-28 00:09:53 +0000 UTC - event for webserver-deployment-dd94f59b7-wnjgv: {kubelet node1} SandboxChanged: Pod sandbox changed, it will be killed and re-created. Aug 28 00:14:44.810: INFO: At 2021-08-28 00:09:54 +0000 UTC - event for webserver-deployment-dd94f59b7-2z9dr: {kubelet node2} SandboxChanged: Pod sandbox changed, it will be killed and re-created. Aug 28 00:14:44.810: INFO: At 2021-08-28 00:09:54 +0000 UTC - event for webserver-deployment-dd94f59b7-f4kxs: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/httpd:2.4.38-alpine" Aug 28 00:14:44.810: INFO: At 2021-08-28 00:09:54 +0000 UTC - event for webserver-deployment-dd94f59b7-f4kxs: {multus } AddedInterface: Add eth0 [10.244.3.90/24] Aug 28 00:14:44.810: INFO: At 2021-08-28 00:09:54 +0000 UTC - event for webserver-deployment-dd94f59b7-f4kxs: {kubelet node1} Failed: Error: ImagePullBackOff Aug 28 00:14:44.810: INFO: At 2021-08-28 00:09:54 +0000 UTC - event for webserver-deployment-dd94f59b7-ntnc5: {kubelet node2} BackOff: Back-off pulling image "docker.io/library/httpd:2.4.38-alpine" Aug 28 00:14:44.810: INFO: At 2021-08-28 00:09:54 +0000 UTC - event for webserver-deployment-dd94f59b7-ntnc5: {kubelet node2} Failed: Error: ImagePullBackOff Aug 28 00:14:44.810: INFO: At 2021-08-28 00:09:54 +0000 UTC - event for webserver-deployment-dd94f59b7-ntnc5: {multus } AddedInterface: Add eth0 [10.244.4.141/24] Aug 28 00:14:44.810: INFO: At 2021-08-28 00:09:55 +0000 UTC - event for webserver-deployment-dd94f59b7-4jn4k: {kubelet node2} Failed: Error: ImagePullBackOff Aug 28 00:14:44.810: INFO: At 2021-08-28 00:09:55 +0000 UTC - event for webserver-deployment-dd94f59b7-4jn4k: {kubelet node2} BackOff: Back-off pulling image "docker.io/library/httpd:2.4.38-alpine" Aug 28 00:14:44.810: INFO: At 2021-08-28 00:09:55 +0000 UTC - event for webserver-deployment-dd94f59b7-4jn4k: {multus } AddedInterface: Add eth0 [10.244.4.142/24] Aug 28 00:14:44.810: INFO: At 2021-08-28 00:09:55 +0000 UTC - event for webserver-deployment-dd94f59b7-77lh5: {kubelet node2} SandboxChanged: Pod sandbox changed, it will be killed and re-created. Aug 28 00:14:44.810: INFO: At 2021-08-28 00:09:55 +0000 UTC - event for webserver-deployment-dd94f59b7-wnjgv: {multus } AddedInterface: Add eth0 [10.244.3.91/24] Aug 28 00:14:44.810: INFO: At 2021-08-28 00:09:55 +0000 UTC - event for webserver-deployment-dd94f59b7-wnjgv: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/httpd:2.4.38-alpine" Aug 28 00:14:44.810: INFO: At 2021-08-28 00:09:55 +0000 UTC - event for webserver-deployment-dd94f59b7-wnjgv: {kubelet node1} Failed: Error: ImagePullBackOff Aug 28 00:14:44.810: INFO: At 2021-08-28 00:09:57 +0000 UTC - event for webserver-deployment-dd94f59b7-2z9dr: {kubelet node2} BackOff: Back-off pulling image "docker.io/library/httpd:2.4.38-alpine" Aug 28 00:14:44.810: INFO: At 2021-08-28 00:09:57 +0000 UTC - event for webserver-deployment-dd94f59b7-2z9dr: {kubelet node2} Failed: Error: ImagePullBackOff Aug 28 00:14:44.810: INFO: At 2021-08-28 00:09:57 +0000 UTC - event for webserver-deployment-dd94f59b7-2z9dr: {multus } AddedInterface: Add eth0 [10.244.4.143/24] Aug 28 00:14:44.810: INFO: At 2021-08-28 00:09:58 +0000 UTC - event for webserver-deployment-dd94f59b7-77lh5: {multus } AddedInterface: Add eth0 [10.244.4.144/24] Aug 28 00:14:44.810: INFO: At 2021-08-28 00:09:58 +0000 UTC - event for webserver-deployment-dd94f59b7-77lh5: {kubelet node2} Failed: Error: ImagePullBackOff Aug 28 00:14:44.810: INFO: At 2021-08-28 00:09:58 +0000 UTC - event for webserver-deployment-dd94f59b7-77lh5: {kubelet node2} BackOff: Back-off pulling image "docker.io/library/httpd:2.4.38-alpine" Aug 28 00:14:44.810: INFO: At 2021-08-28 00:09:58 +0000 UTC - event for webserver-deployment-dd94f59b7-wnjgv: {multus } AddedInterface: Add eth0 [10.244.3.92/24] Aug 28 00:14:44.810: INFO: At 2021-08-28 00:10:00 +0000 UTC - event for webserver-deployment-dd94f59b7-77lh5: {multus } AddedInterface: Add eth0 [10.244.4.145/24] Aug 28 00:14:44.814: INFO: POD NODE PHASE GRACE CONDITIONS Aug 28 00:14:44.814: INFO: webserver-deployment-dd94f59b7-2z9dr node2 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC }] Aug 28 00:14:44.814: INFO: webserver-deployment-dd94f59b7-4jn4k node2 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC }] Aug 28 00:14:44.814: INFO: webserver-deployment-dd94f59b7-56q7x node1 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC }] Aug 28 00:14:44.814: INFO: webserver-deployment-dd94f59b7-77lh5 node2 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC }] Aug 28 00:14:44.814: INFO: webserver-deployment-dd94f59b7-bv7w9 node1 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC }] Aug 28 00:14:44.814: INFO: webserver-deployment-dd94f59b7-f4kxs node1 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC }] Aug 28 00:14:44.814: INFO: webserver-deployment-dd94f59b7-hddhg node1 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC }] Aug 28 00:14:44.814: INFO: webserver-deployment-dd94f59b7-ntnc5 node2 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC }] Aug 28 00:14:44.814: INFO: webserver-deployment-dd94f59b7-rl54s node2 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC }] Aug 28 00:14:44.814: INFO: webserver-deployment-dd94f59b7-wnjgv node1 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 00:09:42 +0000 UTC }] Aug 28 00:14:44.814: INFO: Aug 28 00:14:44.818: INFO: Logging node info for node master1 Aug 28 00:14:44.820: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 3af53387-5aee-42c1-b0e7-644cf9161d48 75501 0 2021-08-27 20:46:13 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"92:f8:b6:72:e4:be"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-08-27 20:46:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-08-27 20:46:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {kube-controller-manager Update v1 2021-08-27 20:46:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:50:56 +0000 UTC,LastTransitionTime:2021-08-27 20:50:56 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 00:14:35 +0000 UTC,LastTransitionTime:2021-08-27 20:46:13 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 00:14:35 +0000 UTC,LastTransitionTime:2021-08-27 20:46:13 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 00:14:35 +0000 UTC,LastTransitionTime:2021-08-27 20:46:13 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 00:14:35 +0000 UTC,LastTransitionTime:2021-08-27 20:50:30 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:5d584023135a46ecb77596bf48ed7f2f,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:587cafa0-6de3-49f8-906e-06315a8ff104,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[sirot/netperf-latest@sha256:23929b922bb077cb341450fc1c90ae42e6f75da5d7ce61cd1880b08560a5ff85 sirot/netperf-latest:latest],SizeBytes:282025213,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:ae72171f047a37ee5423e0692df7429830919af16e9d668ab0c80b723863d102 tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:cd98d1edca8e5e2e3ea42cbc463812483e5d069d10f0974ca9d484b5a7bd68db tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[aquasec/kube-bench@sha256:3544f6662feb73d36fdba35b17652e2fd73aae45bd4b60e76d7ab928220b3cc6 aquasec/kube-bench:0.3.1],SizeBytes:19301876,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Aug 28 00:14:44.821: INFO: Logging kubelet events for node master1 Aug 28 00:14:44.822: INFO: Logging pods the kubelet thinks is on node master1 Aug 28 00:14:44.830: INFO: kube-scheduler-master1 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:44.830: INFO: Container kube-scheduler ready: true, restart count 0 Aug 28 00:14:44.830: INFO: kube-flannel-pp7vp started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded) Aug 28 00:14:44.830: INFO: Init container install-cni ready: true, restart count 0 Aug 28 00:14:44.830: INFO: Container kube-flannel ready: true, restart count 3 Aug 28 00:14:44.830: INFO: kube-multus-ds-amd64-sfr9k started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:44.831: INFO: Container kube-multus ready: true, restart count 1 Aug 28 00:14:44.831: INFO: docker-registry-docker-registry-56cbc7bc58-cthtt started at 2021-08-27 20:51:47 +0000 UTC (0+2 container statuses recorded) Aug 28 00:14:44.831: INFO: Container docker-registry ready: true, restart count 0 Aug 28 00:14:44.831: INFO: Container nginx ready: true, restart count 0 Aug 28 00:14:44.831: INFO: kube-apiserver-master1 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:44.831: INFO: Container kube-apiserver ready: true, restart count 0 Aug 28 00:14:44.831: INFO: kube-controller-manager-master1 started at 2021-08-27 20:54:27 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:44.831: INFO: Container kube-controller-manager ready: true, restart count 3 Aug 28 00:14:44.831: INFO: kube-proxy-rb5p6 started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:44.831: INFO: Container kube-proxy ready: true, restart count 1 Aug 28 00:14:44.831: INFO: coredns-7677f9bb54-dwtp5 started at 2021-08-27 20:49:16 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:44.831: INFO: Container coredns ready: true, restart count 1 Aug 28 00:14:44.831: INFO: node-exporter-z2ngr started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded) Aug 28 00:14:44.831: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 28 00:14:44.831: INFO: Container node-exporter ready: true, restart count 0 W0828 00:14:44.842226 26 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. Aug 28 00:14:44.869: INFO: Latency metrics for node master1 Aug 28 00:14:44.869: INFO: Logging node info for node master2 Aug 28 00:14:44.871: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 666e473f-d9e6-4c06-8b56-06474d788f70 75544 0 2021-08-27 20:46:51 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"42:86:ff:30:bd:4d"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-08-27 20:46:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {kubelet Update v1 2021-08-27 20:46:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-08-27 20:48:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-08-27 20:55:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:51:53 +0000 UTC,LastTransitionTime:2021-08-27 20:51:53 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 00:14:44 +0000 UTC,LastTransitionTime:2021-08-27 20:46:51 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 00:14:44 +0000 UTC,LastTransitionTime:2021-08-27 20:46:51 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 00:14:44 +0000 UTC,LastTransitionTime:2021-08-27 20:46:51 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 00:14:44 +0000 UTC,LastTransitionTime:2021-08-27 20:48:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:2835065974a64998811b9acd85de209b,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:309b9155-1e2a-4ebd-900f-bba5abfc3a5d,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[sirot/netperf-latest@sha256:23929b922bb077cb341450fc1c90ae42e6f75da5d7ce61cd1880b08560a5ff85 sirot/netperf-latest:latest],SizeBytes:282025213,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Aug 28 00:14:44.871: INFO: Logging kubelet events for node master2 Aug 28 00:14:44.874: INFO: Logging pods the kubelet thinks is on node master2 Aug 28 00:14:44.882: INFO: kube-controller-manager-master2 started at 2021-08-27 20:51:01 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:44.882: INFO: Container kube-controller-manager ready: true, restart count 2 Aug 28 00:14:44.882: INFO: kube-flannel-4znnq started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded) Aug 28 00:14:44.882: INFO: Init container install-cni ready: true, restart count 2 Aug 28 00:14:44.882: INFO: Container kube-flannel ready: true, restart count 2 Aug 28 00:14:44.882: INFO: kube-multus-ds-amd64-4mgbk started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:44.882: INFO: Container kube-multus ready: true, restart count 1 Aug 28 00:14:44.882: INFO: prometheus-operator-5bb8cb9d8f-whr5p started at 2021-08-27 20:59:06 +0000 UTC (0+2 container statuses recorded) Aug 28 00:14:44.883: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 28 00:14:44.883: INFO: Container prometheus-operator ready: true, restart count 0 Aug 28 00:14:44.883: INFO: node-exporter-96jk5 started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded) Aug 28 00:14:44.883: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 28 00:14:44.883: INFO: Container node-exporter ready: true, restart count 0 Aug 28 00:14:44.883: INFO: kube-apiserver-master2 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:44.883: INFO: Container kube-apiserver ready: true, restart count 0 Aug 28 00:14:44.883: INFO: kube-scheduler-master2 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:44.883: INFO: Container kube-scheduler ready: true, restart count 2 Aug 28 00:14:44.883: INFO: kube-proxy-b4mn9 started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:44.883: INFO: Container kube-proxy ready: true, restart count 1 Aug 28 00:14:44.883: INFO: node-feature-discovery-controller-5bf5c49849-zr9zd started at 2021-08-27 20:55:09 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:44.883: INFO: Container nfd-controller ready: true, restart count 0 W0828 00:14:44.895724 26 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. Aug 28 00:14:44.923: INFO: Latency metrics for node master2 Aug 28 00:14:44.923: INFO: Logging node info for node master3 Aug 28 00:14:44.926: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 620b6dba-f2c5-46e9-b2ff-d2f4197167d0 75537 0 2021-08-27 20:47:02 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ce:de:ea:c3:40:4f"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-08-27 20:47:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-08-27 20:47:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-08-27 20:48:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:51:44 +0000 UTC,LastTransitionTime:2021-08-27 20:51:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 00:14:42 +0000 UTC,LastTransitionTime:2021-08-27 20:47:02 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 00:14:42 +0000 UTC,LastTransitionTime:2021-08-27 20:47:02 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 00:14:42 +0000 UTC,LastTransitionTime:2021-08-27 20:47:02 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 00:14:42 +0000 UTC,LastTransitionTime:2021-08-27 20:48:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:37decbffe0e84048b5801289ad3be5bf,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:c96fe8b9-1ce0-44cd-935a-b58987e26570,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496891,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Aug 28 00:14:44.926: INFO: Logging kubelet events for node master3 Aug 28 00:14:44.928: INFO: Logging pods the kubelet thinks is on node master3 Aug 28 00:14:44.936: INFO: kube-controller-manager-master3 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:44.936: INFO: Container kube-controller-manager ready: true, restart count 2 Aug 28 00:14:44.936: INFO: kube-proxy-8sxhm started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:44.936: INFO: Container kube-proxy ready: true, restart count 1 Aug 28 00:14:44.936: INFO: kube-flannel-fkz5d started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded) Aug 28 00:14:44.936: INFO: Init container install-cni ready: true, restart count 0 Aug 28 00:14:44.936: INFO: Container kube-flannel ready: true, restart count 1 Aug 28 00:14:44.936: INFO: dns-autoscaler-5b7b5c9b6f-54xch started at 2021-08-27 20:49:19 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:44.936: INFO: Container autoscaler ready: true, restart count 1 Aug 28 00:14:44.936: INFO: coredns-7677f9bb54-rxplt started at 2021-08-27 20:49:21 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:44.936: INFO: Container coredns ready: true, restart count 1 Aug 28 00:14:44.936: INFO: kube-apiserver-master3 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:44.936: INFO: Container kube-apiserver ready: true, restart count 0 Aug 28 00:14:44.936: INFO: kube-scheduler-master3 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:44.936: INFO: Container kube-scheduler ready: true, restart count 2 Aug 28 00:14:44.936: INFO: kube-multus-ds-amd64-wwcgv started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:44.936: INFO: Container kube-multus ready: true, restart count 1 Aug 28 00:14:44.936: INFO: node-exporter-d4m7q started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded) Aug 28 00:14:44.936: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 28 00:14:44.936: INFO: Container node-exporter ready: true, restart count 0 W0828 00:14:44.950136 26 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. Aug 28 00:14:44.973: INFO: Latency metrics for node master3 Aug 28 00:14:44.973: INFO: Logging node info for node node1 Aug 28 00:14:44.975: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 e7a2481e-32db-4c83-bd9f-4a0687258e7a 75531 0 2021-08-27 20:48:09 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.36.2.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"56:c7:37:40:51:ca"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-08-27 20:48:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubeadm Update v1 2021-08-27 20:48:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-08-27 20:55:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-08-27 20:57:40 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-08-27 20:57:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269628928 0} {<nil>} 196552372Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884628480 0} {<nil>} 174692020Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:51:49 +0000 UTC,LastTransitionTime:2021-08-27 20:51:49 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 00:14:41 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 00:14:41 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 00:14:41 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 00:14:41 +0000 UTC,LastTransitionTime:2021-08-27 20:48:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:c1e38e80ea114a5f96601202301ce842,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:e769e86d-15c0-442c-a93b-bcc6c33ff1cd,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:331c6faa8b0d5ec72cf105e87d35df0a2f2baeec3d6217a51faa73f9460f937f localhost:30500/barometer-collectd:stable],SizeBytes:1238704157,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:a7cea43d9d2f67c38fbf0407786edbe660ee9072945f7bb272b55fd255e8eaca opnfv/barometer-collectd:stable],SizeBytes:1075746799,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:1003787960,},ContainerImage{Names:[localhost:30500/cmk@sha256:fd1487b0c07556a087eff669e70c501a704720dcd53ff75183593de6720585f2 cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[aquasec/kube-hunter@sha256:2be6820bc1d7e0f57193a9a27d5a3e16b2fd93c53747b03ce8ca48c6fc323781 aquasec/kube-hunter:0.3.1],SizeBytes:347611549,},ContainerImage{Names:[golang@sha256:db2475a1dbb2149508e5db31d7d77a75e6600d54be645f37681f03f2762169ba golang:alpine3.12],SizeBytes:301186719,},ContainerImage{Names:[sirot/netperf-latest@sha256:23929b922bb077cb341450fc1c90ae42e6f75da5d7ce61cd1880b08560a5ff85 sirot/netperf-latest:latest],SizeBytes:282025213,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:d7300ccf7ff3e9cea2111d275143b8050618bbc1d1ffe41f46286b1696261243 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44393508,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:de25c7fc6c4f3a27c7f0c2dff454e4671823a34d88abd533f210848d527e0fbb alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Aug 28 00:14:44.976: INFO: Logging kubelet events for node node1 Aug 28 00:14:44.978: INFO: Logging pods the kubelet thinks is on node node1 Aug 28 00:14:44.993: INFO: kube-multus-ds-amd64-nn7bl started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:44.993: INFO: Container kube-multus ready: true, restart count 1 Aug 28 00:14:44.993: INFO: kubernetes-metrics-scraper-678c97765c-gtp5x started at 2021-08-27 20:49:21 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:44.993: INFO: Container kubernetes-metrics-scraper ready: true, restart count 1 Aug 28 00:14:44.993: INFO: cmk-init-discover-node1-spg26 started at 2021-08-27 20:57:37 +0000 UTC (0+3 container statuses recorded) Aug 28 00:14:44.993: INFO: Container discover ready: false, restart count 0 Aug 28 00:14:44.993: INFO: Container init ready: false, restart count 0 Aug 28 00:14:44.993: INFO: Container install ready: false, restart count 0 Aug 28 00:14:44.993: INFO: cmk-jw4m6 started at 2021-08-27 20:58:19 +0000 UTC (0+2 container statuses recorded) Aug 28 00:14:44.993: INFO: Container nodereport ready: true, restart count 0 Aug 28 00:14:44.993: INFO: Container reconcile ready: true, restart count 0 Aug 28 00:14:44.993: INFO: webserver-deployment-dd94f59b7-hddhg started at 2021-08-28 00:09:42 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:44.993: INFO: Container httpd ready: false, restart count 0 Aug 28 00:14:44.993: INFO: nginx-proxy-node1 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:44.993: INFO: Container nginx-proxy ready: true, restart count 2 Aug 28 00:14:44.993: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-9lndx started at 2021-08-27 20:55:51 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:44.993: INFO: Container kube-sriovdp ready: true, restart count 0 Aug 28 00:14:44.993: INFO: webserver-deployment-dd94f59b7-56q7x started at 2021-08-28 00:09:42 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:44.993: INFO: Container httpd ready: false, restart count 0 Aug 28 00:14:44.993: INFO: webserver-deployment-dd94f59b7-f4kxs started at 2021-08-28 00:09:42 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:44.993: INFO: Container httpd ready: false, restart count 0 Aug 28 00:14:44.993: INFO: node-exporter-4cvlq started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded) Aug 28 00:14:44.993: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 28 00:14:44.993: INFO: Container node-exporter ready: true, restart count 0 Aug 28 00:14:44.993: INFO: var-expansion-c588bc90-2b5d-4887-b768-04f3a7490adc started at 2021-08-28 00:11:25 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:44.993: INFO: Container dapi-container ready: false, restart count 0 Aug 28 00:14:44.993: INFO: kubernetes-dashboard-86c6f9df5b-c56fg started at 2021-08-27 20:49:21 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:44.993: INFO: Container kubernetes-dashboard ready: true, restart count 1 Aug 28 00:14:44.993: INFO: node-feature-discovery-worker-bd9kg started at 2021-08-27 20:55:06 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:44.993: INFO: Container nfd-worker ready: true, restart count 0 Aug 28 00:14:44.993: INFO: webserver-deployment-dd94f59b7-bv7w9 started at 2021-08-28 00:09:42 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:44.994: INFO: Container httpd ready: false, restart count 0 Aug 28 00:14:44.994: INFO: kube-proxy-pb5bl started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:44.994: INFO: Container kube-proxy ready: true, restart count 2 Aug 28 00:14:44.994: INFO: collectd-ccvwg started at 2021-08-27 21:04:15 +0000 UTC (0+3 container statuses recorded) Aug 28 00:14:44.994: INFO: Container collectd ready: true, restart count 0 Aug 28 00:14:44.994: INFO: Container collectd-exporter ready: true, restart count 0 Aug 28 00:14:44.994: INFO: Container rbac-proxy ready: true, restart count 0 Aug 28 00:14:44.994: INFO: prometheus-k8s-0 started at 2021-08-27 20:59:29 +0000 UTC (0+5 container statuses recorded) Aug 28 00:14:44.994: INFO: Container custom-metrics-apiserver ready: true, restart count 0 Aug 28 00:14:44.994: INFO: Container grafana ready: true, restart count 0 Aug 28 00:14:44.994: INFO: Container prometheus ready: true, restart count 1 Aug 28 00:14:44.994: INFO: Container prometheus-config-reloader ready: true, restart count 0 Aug 28 00:14:44.994: INFO: Container rules-configmap-reloader ready: true, restart count 0 Aug 28 00:14:44.994: INFO: adopt-release-s6jg4 started at 2021-08-28 00:07:35 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:44.994: INFO: Container c ready: false, restart count 0 Aug 28 00:14:44.994: INFO: webserver-deployment-dd94f59b7-wnjgv started at 2021-08-28 00:09:42 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:44.994: INFO: Container httpd ready: false, restart count 0 Aug 28 00:14:44.994: INFO: kube-flannel-ssxn7 started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded) Aug 28 00:14:44.994: INFO: Init container install-cni ready: true, restart count 0 Aug 28 00:14:44.994: INFO: Container kube-flannel ready: true, restart count 1 Aug 28 00:14:44.994: INFO: e2e-test-httpd-pod started at 2021-08-28 00:13:14 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:44.994: INFO: Container e2e-test-httpd-pod ready: false, restart count 0 W0828 00:14:45.012516 26 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. Aug 28 00:14:45.044: INFO: Latency metrics for node node1 Aug 28 00:14:45.044: INFO: Logging node info for node node2 Aug 28 00:14:45.046: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 eb7da7c0-513f-4072-a078-ad3d24f88114 75506 0 2021-08-27 20:48:09 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.36.2.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"9a:88:9c:d1:39:68"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-08-27 20:48:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubeadm Update v1 2021-08-27 20:48:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-08-27 20:55:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-08-27 20:58:03 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-08-27 20:58:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269628928 0} {<nil>} 196552372Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884628480 0} {<nil>} 174692020Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:50:36 +0000 UTC,LastTransitionTime:2021-08-27 20:50:36 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 00:14:37 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 00:14:37 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 00:14:37 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 00:14:37 +0000 UTC,LastTransitionTime:2021-08-27 20:48:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:0ccfc2a4a9b7400c9ca53b5de0ca4970,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:26b02690-0814-4c92-9f6d-d315df796ce6,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:331c6faa8b0d5ec72cf105e87d35df0a2f2baeec3d6217a51faa73f9460f937f localhost:30500/barometer-collectd:stable],SizeBytes:1238704157,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[localhost:30500/cmk@sha256:fd1487b0c07556a087eff669e70c501a704720dcd53ff75183593de6720585f2 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:d7300ccf7ff3e9cea2111d275143b8050618bbc1d1ffe41f46286b1696261243 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44393508,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:ae72171f047a37ee5423e0692df7429830919af16e9d668ab0c80b723863d102 localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:cd98d1edca8e5e2e3ea42cbc463812483e5d069d10f0974ca9d484b5a7bd68db localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[aquasec/kube-bench@sha256:3544f6662feb73d36fdba35b17652e2fd73aae45bd4b60e76d7ab928220b3cc6 aquasec/kube-bench:0.3.1],SizeBytes:19301876,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Aug 28 00:14:45.047: INFO: Logging kubelet events for node node2 Aug 28 00:14:45.049: INFO: Logging pods the kubelet thinks is on node node2 Aug 28 00:14:45.063: INFO: node-feature-discovery-worker-54lfh started at 2021-08-27 20:55:06 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:45.063: INFO: Container nfd-worker ready: true, restart count 0 Aug 28 00:14:45.063: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4f962 started at 2021-08-27 20:55:51 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:45.063: INFO: Container kube-sriovdp ready: true, restart count 0 Aug 28 00:14:45.063: INFO: webserver-deployment-dd94f59b7-4jn4k started at 2021-08-28 00:09:42 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:45.063: INFO: Container httpd ready: false, restart count 0 Aug 28 00:14:45.063: INFO: nginx-proxy-node2 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:45.063: INFO: Container nginx-proxy ready: true, restart count 1 Aug 28 00:14:45.063: INFO: kube-proxy-r4q4t started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:45.063: INFO: Container kube-proxy ready: true, restart count 2 Aug 28 00:14:45.063: INFO: kube-multus-ds-amd64-tfffk started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:45.063: INFO: Container kube-multus ready: true, restart count 1 Aug 28 00:14:45.063: INFO: node-exporter-p6h5h started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded) Aug 28 00:14:45.063: INFO: Container kube-rbac-proxy ready: true, restart count 0 Aug 28 00:14:45.063: INFO: Container node-exporter ready: true, restart count 0 Aug 28 00:14:45.063: INFO: cmk-fzjgr started at 2021-08-27 20:58:20 +0000 UTC (0+2 container statuses recorded) Aug 28 00:14:45.063: INFO: Container nodereport ready: true, restart count 0 Aug 28 00:14:45.063: INFO: Container reconcile ready: true, restart count 0 Aug 28 00:14:45.063: INFO: webserver-deployment-dd94f59b7-77lh5 started at 2021-08-28 00:09:42 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:45.063: INFO: Container httpd ready: false, restart count 0 Aug 28 00:14:45.063: INFO: adopt-release-thh76 started at 2021-08-28 00:07:35 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:45.063: INFO: Container c ready: false, restart count 0 Aug 28 00:14:45.063: INFO: webserver-deployment-dd94f59b7-rl54s started at 2021-08-28 00:09:42 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:45.063: INFO: Container httpd ready: false, restart count 0 Aug 28 00:14:45.063: INFO: downward-api-c9653810-7bb3-4aad-bed1-588d15cecffc started at 2021-08-28 00:10:18 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:45.063: INFO: Container dapi-container ready: false, restart count 0 Aug 28 00:14:45.063: INFO: webserver-deployment-dd94f59b7-2z9dr started at 2021-08-28 00:09:42 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:45.063: INFO: Container httpd ready: false, restart count 0 Aug 28 00:14:45.063: INFO: webserver-deployment-dd94f59b7-ntnc5 started at 2021-08-28 00:09:42 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:45.063: INFO: Container httpd ready: false, restart count 0 Aug 28 00:14:45.063: INFO: collectd-64dp2 started at 2021-08-27 21:04:15 +0000 UTC (0+3 container statuses recorded) Aug 28 00:14:45.063: INFO: Container collectd ready: true, restart count 0 Aug 28 00:14:45.063: INFO: Container collectd-exporter ready: true, restart count 0 Aug 28 00:14:45.063: INFO: Container rbac-proxy ready: true, restart count 0 Aug 28 00:14:45.063: INFO: affinity-nodeport-transition-zf6q8 started at 2021-08-28 00:12:22 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:45.063: INFO: Container affinity-nodeport-transition ready: false, restart count 0 Aug 28 00:14:45.064: INFO: test-cleanup-controller-gscmn started at 2021-08-28 00:13:33 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:45.064: INFO: Container httpd ready: false, restart count 0 Aug 28 00:14:45.064: INFO: cmk-webhook-6c9d5f8578-ndbx2 started at 2021-08-27 20:58:20 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:45.064: INFO: Container cmk-webhook ready: true, restart count 0 Aug 28 00:14:45.064: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-v99df started at 2021-08-27 21:02:08 +0000 UTC (0+2 container statuses recorded) Aug 28 00:14:45.064: INFO: Container tas-controller ready: true, restart count 0 Aug 28 00:14:45.064: INFO: Container tas-extender ready: true, restart count 0 Aug 28 00:14:45.064: INFO: downward-api-6ebc473f-df4d-4cc6-9ed0-d392e332a3ac started at 2021-08-28 00:12:14 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:45.064: INFO: Container dapi-container ready: false, restart count 0 Aug 28 00:14:45.064: INFO: kube-flannel-t9qv4 started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded) Aug 28 00:14:45.064: INFO: Init container install-cni ready: true, restart count 0 Aug 28 00:14:45.064: INFO: Container kube-flannel ready: true, restart count 2 Aug 28 00:14:45.064: INFO: cmk-init-discover-node2-l9qjd started at 2021-08-27 20:57:57 +0000 UTC (0+3 container statuses recorded) Aug 28 00:14:45.064: INFO: Container discover ready: false, restart count 0 Aug 28 00:14:45.064: INFO: Container init ready: false, restart count 0 Aug 28 00:14:45.064: INFO: Container install ready: false, restart count 0 Aug 28 00:14:45.064: INFO: affinity-nodeport-transition-6z5pl started at 2021-08-28 00:12:22 +0000 UTC (0+1 container statuses recorded) Aug 28 00:14:45.064: INFO: Container affinity-nodeport-transition ready: false, restart count 0 W0828 00:14:45.076631 26 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. Aug 28 00:14:45.335: INFO: Latency metrics for node node2 Aug 28 00:14:45.335: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "deployment-4063" for this suite.