/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:251
Aug 28 02:55:09.967: Unexpected error:
<*errors.errorString | 0xc001c111f0>: {
s: "pod \"pod-ff6602be-7658-4624-bb3a-8efd6aa52c11\" is not Running: timed out waiting for the condition",
}
pod "pod-ff6602be-7658-4624-bb3a-8efd6aa52c11" is not Running: timed out waiting for the condition
occurred
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:764
[BeforeEach] [sig-storage] PersistentVolumes-local
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174
STEP: Creating a kubernetes client
Aug 28 02:51:59.729: INFO: >>> kubeConfig: /root/.kube/config
STEP: Building a namespace api object, basename persistent-local-volumes-test
STEP: Waiting for a default service account to be provisioned in namespace
[BeforeEach] [sig-storage] PersistentVolumes-local
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155
[BeforeEach] [Volume type: dir-bindmounted]
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191
STEP: Initializing test volumes
Aug 28 02:53:07.785: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir /tmp/local-volume-test-84c7f912-0bec-483e-9b6c-53f8917a48e2 && mount --bind /tmp/local-volume-test-84c7f912-0bec-483e-9b6c-53f8917a48e2 /tmp/local-volume-test-84c7f912-0bec-483e-9b6c-53f8917a48e2] Namespace:persistent-local-volumes-test-96 PodName:hostexec-node1-wgm8c ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 02:53:07.785: INFO: >>> kubeConfig: /root/.kube/config
STEP: Creating local PVCs and PVs
Aug 28 02:53:07.924: INFO: Creating a PV followed by a PVC
Aug 28 02:53:07.936: INFO: Waiting for PV local-pvvzjs5 to bind to PVC pvc-n465j
Aug 28 02:53:07.936: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-n465j] to have phase Bound
Aug 28 02:53:07.938: INFO: PersistentVolumeClaim pvc-n465j found but phase is Pending instead of Bound.
Aug 28 02:53:09.941: INFO: PersistentVolumeClaim pvc-n465j found and phase=Bound (2.004473568s)
Aug 28 02:53:09.941: INFO: Waiting up to 3m0s for PersistentVolume local-pvvzjs5 to have phase Bound
Aug 28 02:53:09.943: INFO: PersistentVolume local-pvvzjs5 found and phase=Bound (1.887036ms)
[It] should be able to write from pod1 and read from pod2
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:251
STEP: Creating pod1
STEP: Creating a pod
Aug 28 02:55:09.967: FAIL: Unexpected error:
<*errors.errorString | 0xc001c111f0>: {
s: "pod \"pod-ff6602be-7658-4624-bb3a-8efd6aa52c11\" is not Running: timed out waiting for the condition",
}
pod "pod-ff6602be-7658-4624-bb3a-8efd6aa52c11" is not Running: timed out waiting for the condition
occurred
Full Stack Trace
k8s.io/kubernetes/test/e2e/storage.twoPodsReadWriteSerialTest(0xc002b50c60, 0xc0006a1180, 0xc003d72510)
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:764 +0xaf
k8s.io/kubernetes/test/e2e/storage.glob..func20.2.5.1()
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:252 +0x45
k8s.io/kubernetes/test/e2e.RunE2ETests(0xc0038dcc00)
_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345
k8s.io/kubernetes/test/e2e.TestE2E(0xc0038dcc00)
_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b
testing.tRunner(0xc0038dcc00, 0x4dec428)
/usr/local/go/src/testing/testing.go:1123 +0xef
created by testing.(*T).Run
/usr/local/go/src/testing/testing.go:1168 +0x2b3
[AfterEach] [Volume type: dir-bindmounted]
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200
STEP: Cleaning up PVC and PV
Aug 28 02:55:09.969: INFO: Deleting PersistentVolumeClaim "pvc-n465j"
Aug 28 02:55:09.977: INFO: Deleting PersistentVolume "local-pvvzjs5"
STEP: Removing the test directory
Aug 28 02:55:09.983: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c umount /tmp/local-volume-test-84c7f912-0bec-483e-9b6c-53f8917a48e2 && rm -r /tmp/local-volume-test-84c7f912-0bec-483e-9b6c-53f8917a48e2] Namespace:persistent-local-volumes-test-96 PodName:hostexec-node1-wgm8c ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 02:55:09.983: INFO: >>> kubeConfig: /root/.kube/config
[AfterEach] [sig-storage] PersistentVolumes-local
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175
STEP: Collecting events from namespace "persistent-local-volumes-test-96".
STEP: Found 14 events.
Aug 28 02:55:10.133: INFO: At 2021-08-28 02:51:59 +0000 UTC - event for hostexec-node1-wgm8c: {default-scheduler } Scheduled: Successfully assigned persistent-local-volumes-test-96/hostexec-node1-wgm8c to node1
Aug 28 02:55:10.133: INFO: At 2021-08-28 02:52:36 +0000 UTC - event for hostexec-node1-wgm8c: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20"
Aug 28 02:55:10.133: INFO: At 2021-08-28 02:52:37 +0000 UTC - event for hostexec-node1-wgm8c: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 495.662964ms
Aug 28 02:55:10.133: INFO: At 2021-08-28 02:52:37 +0000 UTC - event for hostexec-node1-wgm8c: {kubelet node1} Created: Created container agnhost-container
Aug 28 02:55:10.133: INFO: At 2021-08-28 02:52:37 +0000 UTC - event for hostexec-node1-wgm8c: {kubelet node1} Started: Started container agnhost-container
Aug 28 02:55:10.133: INFO: At 2021-08-28 02:53:09 +0000 UTC - event for pod-ff6602be-7658-4624-bb3a-8efd6aa52c11: {default-scheduler } Scheduled: Successfully assigned persistent-local-volumes-test-96/pod-ff6602be-7658-4624-bb3a-8efd6aa52c11 to node1
Aug 28 02:55:10.133: INFO: At 2021-08-28 02:53:12 +0000 UTC - event for pod-ff6602be-7658-4624-bb3a-8efd6aa52c11: {multus } AddedInterface: Add eth0 [10.244.3.136/24]
Aug 28 02:55:10.133: INFO: At 2021-08-28 02:53:12 +0000 UTC - event for pod-ff6602be-7658-4624-bb3a-8efd6aa52c11: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29"
Aug 28 02:55:10.133: INFO: At 2021-08-28 02:53:13 +0000 UTC - event for pod-ff6602be-7658-4624-bb3a-8efd6aa52c11: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit
Aug 28 02:55:10.133: INFO: At 2021-08-28 02:53:13 +0000 UTC - event for pod-ff6602be-7658-4624-bb3a-8efd6aa52c11: {kubelet node1} Failed: Error: ErrImagePull
Aug 28 02:55:10.133: INFO: At 2021-08-28 02:53:14 +0000 UTC - event for pod-ff6602be-7658-4624-bb3a-8efd6aa52c11: {kubelet node1} SandboxChanged: Pod sandbox changed, it will be killed and re-created.
Aug 28 02:55:10.133: INFO: At 2021-08-28 02:53:16 +0000 UTC - event for pod-ff6602be-7658-4624-bb3a-8efd6aa52c11: {multus } AddedInterface: Add eth0 [10.244.3.137/24]
Aug 28 02:55:10.133: INFO: At 2021-08-28 02:53:16 +0000 UTC - event for pod-ff6602be-7658-4624-bb3a-8efd6aa52c11: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29"
Aug 28 02:55:10.133: INFO: At 2021-08-28 02:53:16 +0000 UTC - event for pod-ff6602be-7658-4624-bb3a-8efd6aa52c11: {kubelet node1} Failed: Error: ImagePullBackOff
Aug 28 02:55:10.136: INFO: POD NODE PHASE GRACE CONDITIONS
Aug 28 02:55:10.136: INFO: hostexec-node1-wgm8c node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 02:51:59 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 02:52:38 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 02:52:38 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 02:51:59 +0000 UTC }]
Aug 28 02:55:10.136: INFO: pod-ff6602be-7658-4624-bb3a-8efd6aa52c11 node1 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 02:53:09 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-08-28 02:53:09 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-08-28 02:53:09 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 02:53:09 +0000 UTC }]
Aug 28 02:55:10.136: INFO:
Aug 28 02:55:10.140: INFO:
Logging node info for node master1
Aug 28 02:55:10.143: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 3af53387-5aee-42c1-b0e7-644cf9161d48 133982 0 2021-08-27 20:46:13 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"92:f8:b6:72:e4:be"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-08-27 20:46:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-08-27 20:46:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {kube-controller-manager Update v1 2021-08-27 20:46:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:50:56 +0000 UTC,LastTransitionTime:2021-08-27 20:50:56 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 02:55:03 +0000 UTC,LastTransitionTime:2021-08-27 20:46:13 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 02:55:03 +0000 UTC,LastTransitionTime:2021-08-27 20:46:13 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 02:55:03 +0000 UTC,LastTransitionTime:2021-08-27 20:46:13 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 02:55:03 +0000 UTC,LastTransitionTime:2021-08-27 20:50:30 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:5d584023135a46ecb77596bf48ed7f2f,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:587cafa0-6de3-49f8-906e-06315a8ff104,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[sirot/netperf-latest@sha256:23929b922bb077cb341450fc1c90ae42e6f75da5d7ce61cd1880b08560a5ff85 sirot/netperf-latest:latest],SizeBytes:282025213,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:ae72171f047a37ee5423e0692df7429830919af16e9d668ab0c80b723863d102 tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:cd98d1edca8e5e2e3ea42cbc463812483e5d069d10f0974ca9d484b5a7bd68db tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[aquasec/kube-bench@sha256:3544f6662feb73d36fdba35b17652e2fd73aae45bd4b60e76d7ab928220b3cc6 aquasec/kube-bench:0.3.1],SizeBytes:19301876,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Aug 28 02:55:10.143: INFO:
Logging kubelet events for node master1
Aug 28 02:55:10.146: INFO:
Logging pods the kubelet thinks is on node master1
Aug 28 02:55:10.164: INFO: kube-scheduler-master1 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.164: INFO: Container kube-scheduler ready: true, restart count 0
Aug 28 02:55:10.164: INFO: kube-flannel-pp7vp started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded)
Aug 28 02:55:10.164: INFO: Init container install-cni ready: true, restart count 0
Aug 28 02:55:10.164: INFO: Container kube-flannel ready: true, restart count 3
Aug 28 02:55:10.164: INFO: kube-multus-ds-amd64-sfr9k started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.164: INFO: Container kube-multus ready: true, restart count 1
Aug 28 02:55:10.164: INFO: docker-registry-docker-registry-56cbc7bc58-cthtt started at 2021-08-27 20:51:47 +0000 UTC (0+2 container statuses recorded)
Aug 28 02:55:10.164: INFO: Container docker-registry ready: true, restart count 0
Aug 28 02:55:10.164: INFO: Container nginx ready: true, restart count 0
Aug 28 02:55:10.164: INFO: node-exporter-z2ngr started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded)
Aug 28 02:55:10.164: INFO: Container kube-rbac-proxy ready: true, restart count 0
Aug 28 02:55:10.164: INFO: Container node-exporter ready: true, restart count 0
Aug 28 02:55:10.164: INFO: kube-apiserver-master1 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.164: INFO: Container kube-apiserver ready: true, restart count 0
Aug 28 02:55:10.164: INFO: kube-controller-manager-master1 started at 2021-08-27 20:54:27 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.164: INFO: Container kube-controller-manager ready: true, restart count 3
Aug 28 02:55:10.164: INFO: kube-proxy-rb5p6 started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.164: INFO: Container kube-proxy ready: true, restart count 1
Aug 28 02:55:10.164: INFO: coredns-7677f9bb54-dwtp5 started at 2021-08-27 20:49:16 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.164: INFO: Container coredns ready: true, restart count 1
W0828 02:55:10.178037 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled.
Aug 28 02:55:10.207: INFO:
Latency metrics for node master1
Aug 28 02:55:10.207: INFO:
Logging node info for node master2
Aug 28 02:55:10.209: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 666e473f-d9e6-4c06-8b56-06474d788f70 133971 0 2021-08-27 20:46:51 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"42:86:ff:30:bd:4d"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-08-27 20:46:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {kubelet Update v1 2021-08-27 20:46:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-08-27 20:48:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-08-27 20:55:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:51:53 +0000 UTC,LastTransitionTime:2021-08-27 20:51:53 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 02:55:02 +0000 UTC,LastTransitionTime:2021-08-27 20:46:51 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 02:55:02 +0000 UTC,LastTransitionTime:2021-08-27 20:46:51 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 02:55:02 +0000 UTC,LastTransitionTime:2021-08-27 20:46:51 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 02:55:02 +0000 UTC,LastTransitionTime:2021-08-27 20:48:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:2835065974a64998811b9acd85de209b,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:309b9155-1e2a-4ebd-900f-bba5abfc3a5d,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[sirot/netperf-latest@sha256:23929b922bb077cb341450fc1c90ae42e6f75da5d7ce61cd1880b08560a5ff85 sirot/netperf-latest:latest],SizeBytes:282025213,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Aug 28 02:55:10.210: INFO:
Logging kubelet events for node master2
Aug 28 02:55:10.212: INFO:
Logging pods the kubelet thinks is on node master2
Aug 28 02:55:10.220: INFO: kube-apiserver-master2 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.220: INFO: Container kube-apiserver ready: true, restart count 0
Aug 28 02:55:10.220: INFO: kube-scheduler-master2 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.220: INFO: Container kube-scheduler ready: true, restart count 2
Aug 28 02:55:10.220: INFO: kube-proxy-b4mn9 started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.220: INFO: Container kube-proxy ready: true, restart count 1
Aug 28 02:55:10.220: INFO: node-feature-discovery-controller-5bf5c49849-zr9zd started at 2021-08-27 20:55:09 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.220: INFO: Container nfd-controller ready: true, restart count 0
Aug 28 02:55:10.220: INFO: kube-controller-manager-master2 started at 2021-08-27 20:51:01 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.220: INFO: Container kube-controller-manager ready: true, restart count 2
Aug 28 02:55:10.220: INFO: kube-flannel-4znnq started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded)
Aug 28 02:55:10.220: INFO: Init container install-cni ready: true, restart count 2
Aug 28 02:55:10.220: INFO: Container kube-flannel ready: true, restart count 2
Aug 28 02:55:10.220: INFO: kube-multus-ds-amd64-4mgbk started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.220: INFO: Container kube-multus ready: true, restart count 1
Aug 28 02:55:10.220: INFO: prometheus-operator-5bb8cb9d8f-whr5p started at 2021-08-27 20:59:06 +0000 UTC (0+2 container statuses recorded)
Aug 28 02:55:10.220: INFO: Container kube-rbac-proxy ready: true, restart count 0
Aug 28 02:55:10.220: INFO: Container prometheus-operator ready: true, restart count 0
Aug 28 02:55:10.220: INFO: node-exporter-96jk5 started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded)
Aug 28 02:55:10.220: INFO: Container kube-rbac-proxy ready: true, restart count 0
Aug 28 02:55:10.220: INFO: Container node-exporter ready: true, restart count 0
W0828 02:55:10.231977 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled.
Aug 28 02:55:10.256: INFO:
Latency metrics for node master2
Aug 28 02:55:10.256: INFO:
Logging node info for node master3
Aug 28 02:55:10.258: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 620b6dba-f2c5-46e9-b2ff-d2f4197167d0 133960 0 2021-08-27 20:47:02 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ce:de:ea:c3:40:4f"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-08-27 20:47:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-08-27 20:47:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-08-27 20:48:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:51:44 +0000 UTC,LastTransitionTime:2021-08-27 20:51:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 02:55:00 +0000 UTC,LastTransitionTime:2021-08-27 20:47:02 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 02:55:00 +0000 UTC,LastTransitionTime:2021-08-27 20:47:02 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 02:55:00 +0000 UTC,LastTransitionTime:2021-08-27 20:47:02 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 02:55:00 +0000 UTC,LastTransitionTime:2021-08-27 20:48:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:37decbffe0e84048b5801289ad3be5bf,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:c96fe8b9-1ce0-44cd-935a-b58987e26570,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496891,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Aug 28 02:55:10.258: INFO:
Logging kubelet events for node master3
Aug 28 02:55:10.261: INFO:
Logging pods the kubelet thinks is on node master3
Aug 28 02:55:10.270: INFO: kube-apiserver-master3 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.270: INFO: Container kube-apiserver ready: true, restart count 0
Aug 28 02:55:10.270: INFO: kube-scheduler-master3 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.270: INFO: Container kube-scheduler ready: true, restart count 2
Aug 28 02:55:10.270: INFO: kube-multus-ds-amd64-wwcgv started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.270: INFO: Container kube-multus ready: true, restart count 1
Aug 28 02:55:10.270: INFO: node-exporter-d4m7q started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded)
Aug 28 02:55:10.270: INFO: Container kube-rbac-proxy ready: true, restart count 0
Aug 28 02:55:10.270: INFO: Container node-exporter ready: true, restart count 0
Aug 28 02:55:10.270: INFO: kube-controller-manager-master3 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.270: INFO: Container kube-controller-manager ready: true, restart count 2
Aug 28 02:55:10.270: INFO: kube-proxy-8sxhm started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.270: INFO: Container kube-proxy ready: true, restart count 1
Aug 28 02:55:10.270: INFO: kube-flannel-fkz5d started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded)
Aug 28 02:55:10.270: INFO: Init container install-cni ready: true, restart count 0
Aug 28 02:55:10.270: INFO: Container kube-flannel ready: true, restart count 1
Aug 28 02:55:10.270: INFO: dns-autoscaler-5b7b5c9b6f-54xch started at 2021-08-27 20:49:19 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.270: INFO: Container autoscaler ready: true, restart count 1
Aug 28 02:55:10.270: INFO: coredns-7677f9bb54-rxplt started at 2021-08-27 20:49:21 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.270: INFO: Container coredns ready: true, restart count 1
W0828 02:55:10.283700 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled.
Aug 28 02:55:10.308: INFO:
Latency metrics for node master3
Aug 28 02:55:10.308: INFO:
Logging node info for node node1
Aug 28 02:55:10.311: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 e7a2481e-32db-4c83-bd9f-4a0687258e7a 133995 0 2021-08-27 20:48:09 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.36.2.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1844":"csi-mock-csi-mock-volumes-1844","csi-mock-csi-mock-volumes-6248":"csi-mock-csi-mock-volumes-6248","csi-mock-csi-mock-volumes-7866":"csi-mock-csi-mock-volumes-7866","csi-mock-csi-mock-volumes-9165":"csi-mock-csi-mock-volumes-9165","csi-mock-csi-mock-volumes-9410":"csi-mock-csi-mock-volumes-9410"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"56:c7:37:40:51:ca"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-08-27 20:48:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-08-27 20:55:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-08-27 20:57:40 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-08-28 02:45:58 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-08-28 02:53:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubelet Update v1 2021-08-28 02:54:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:io.kubernetes.storage.mock/node":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {<nil>} 10 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269628928 0} {<nil>} 196552372Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {<nil>} 10 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884628480 0} {<nil>} 174692020Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:51:49 +0000 UTC,LastTransitionTime:2021-08-27 20:51:49 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 02:55:05 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 02:55:05 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 02:55:05 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 02:55:05 +0000 UTC,LastTransitionTime:2021-08-27 20:48:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:c1e38e80ea114a5f96601202301ce842,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:e769e86d-15c0-442c-a93b-bcc6c33ff1cd,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:331c6faa8b0d5ec72cf105e87d35df0a2f2baeec3d6217a51faa73f9460f937f localhost:30500/barometer-collectd:stable],SizeBytes:1238704157,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:a7cea43d9d2f67c38fbf0407786edbe660ee9072945f7bb272b55fd255e8eaca opnfv/barometer-collectd:stable],SizeBytes:1075746799,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:1003787960,},ContainerImage{Names:[localhost:30500/cmk@sha256:fd1487b0c07556a087eff669e70c501a704720dcd53ff75183593de6720585f2 cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[aquasec/kube-hunter@sha256:2be6820bc1d7e0f57193a9a27d5a3e16b2fd93c53747b03ce8ca48c6fc323781 aquasec/kube-hunter:0.3.1],SizeBytes:347611549,},ContainerImage{Names:[golang@sha256:db2475a1dbb2149508e5db31d7d77a75e6600d54be645f37681f03f2762169ba golang:alpine3.12],SizeBytes:301186719,},ContainerImage{Names:[sirot/netperf-latest@sha256:23929b922bb077cb341450fc1c90ae42e6f75da5d7ce61cd1880b08560a5ff85 sirot/netperf-latest:latest],SizeBytes:282025213,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:d7300ccf7ff3e9cea2111d275143b8050618bbc1d1ffe41f46286b1696261243 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44393508,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/regression-issue-74839-amd64@sha256:3b36bd80b97c532a774e7f6246797b8575d97037982f353476c703ba6686c75c gcr.io/kubernetes-e2e-test-images/regression-issue-74839-amd64:1.0],SizeBytes:19227369,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:de25c7fc6c4f3a27c7f0c2dff454e4671823a34d88abd533f210848d527e0fbb alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Aug 28 02:55:10.312: INFO:
Logging kubelet events for node node1
Aug 28 02:55:10.314: INFO:
Logging pods the kubelet thinks is on node node1
Aug 28 02:55:10.333: INFO: pod-ff6602be-7658-4624-bb3a-8efd6aa52c11 started at 2021-08-28 02:53:09 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.333: INFO: Container write-pod ready: false, restart count 0
Aug 28 02:55:10.333: INFO: pod-11ce3543-1093-4641-b32e-d42ff5de2b23 started at 2021-08-28 02:53:52 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.333: INFO: Container write-pod ready: false, restart count 0
Aug 28 02:55:10.333: INFO: hostexec-node1-46bf7 started at 2021-08-28 02:54:23 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.333: INFO: Container agnhost-container ready: true, restart count 0
Aug 28 02:55:10.333: INFO: kube-proxy-pb5bl started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.333: INFO: Container kube-proxy ready: true, restart count 2
Aug 28 02:55:10.333: INFO: collectd-ccvwg started at 2021-08-27 21:04:15 +0000 UTC (0+3 container statuses recorded)
Aug 28 02:55:10.333: INFO: Container collectd ready: true, restart count 0
Aug 28 02:55:10.333: INFO: Container collectd-exporter ready: true, restart count 0
Aug 28 02:55:10.333: INFO: Container rbac-proxy ready: true, restart count 0
Aug 28 02:55:10.333: INFO: hostexec-node1-4jj4t started at 2021-08-28 02:53:38 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.333: INFO: Container agnhost-container ready: true, restart count 0
Aug 28 02:55:10.333: INFO: kube-flannel-ssxn7 started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded)
Aug 28 02:55:10.333: INFO: Init container install-cni ready: true, restart count 0
Aug 28 02:55:10.333: INFO: Container kube-flannel ready: true, restart count 1
Aug 28 02:55:10.333: INFO: hostexec-node1-8mc4z started at 2021-08-28 02:53:19 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.333: INFO: Container agnhost-container ready: true, restart count 0
Aug 28 02:55:10.333: INFO: cmk-jw4m6 started at 2021-08-27 20:58:19 +0000 UTC (0+2 container statuses recorded)
Aug 28 02:55:10.333: INFO: Container nodereport ready: true, restart count 0
Aug 28 02:55:10.333: INFO: Container reconcile ready: true, restart count 0
Aug 28 02:55:10.333: INFO: hostexec-node1-wgm8c started at 2021-08-28 02:51:59 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.333: INFO: Container agnhost-container ready: true, restart count 0
Aug 28 02:55:10.333: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-9lndx started at 2021-08-27 20:55:51 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.333: INFO: Container kube-sriovdp ready: true, restart count 0
Aug 28 02:55:10.333: INFO: hostexec-node1-9k8hc started at 2021-08-28 02:53:32 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.333: INFO: Container agnhost-container ready: true, restart count 0
Aug 28 02:55:10.333: INFO: node-exporter-4cvlq started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded)
Aug 28 02:55:10.333: INFO: Container kube-rbac-proxy ready: true, restart count 0
Aug 28 02:55:10.333: INFO: Container node-exporter ready: true, restart count 0
Aug 28 02:55:10.333: INFO: pod-7f61b887-e4ff-47f3-ac7f-1b0c3542c2e5 started at 2021-08-28 02:54:23 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.333: INFO: Container write-pod ready: false, restart count 0
Aug 28 02:55:10.333: INFO: kubernetes-dashboard-86c6f9df5b-c56fg started at 2021-08-27 20:49:21 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.333: INFO: Container kubernetes-dashboard ready: true, restart count 1
Aug 28 02:55:10.333: INFO: node-feature-discovery-worker-bd9kg started at 2021-08-27 20:55:06 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.333: INFO: Container nfd-worker ready: true, restart count 0
Aug 28 02:55:10.333: INFO: prometheus-k8s-0 started at 2021-08-27 20:59:29 +0000 UTC (0+5 container statuses recorded)
Aug 28 02:55:10.333: INFO: Container custom-metrics-apiserver ready: true, restart count 0
Aug 28 02:55:10.333: INFO: Container grafana ready: true, restart count 0
Aug 28 02:55:10.333: INFO: Container prometheus ready: true, restart count 1
Aug 28 02:55:10.333: INFO: Container prometheus-config-reloader ready: true, restart count 0
Aug 28 02:55:10.333: INFO: Container rules-configmap-reloader ready: true, restart count 0
Aug 28 02:55:10.333: INFO: hostexec-node1-lxk5h started at 2021-08-28 02:54:32 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.333: INFO: Container agnhost-container ready: true, restart count 0
Aug 28 02:55:10.333: INFO: pod-9ff27f8e-2251-49c3-8591-baa4b3fe8f3b started at 2021-08-28 02:54:38 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.333: INFO: Container write-pod ready: false, restart count 0
Aug 28 02:55:10.333: INFO: hostexec-node1-g44dj started at 2021-08-28 02:54:16 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.333: INFO: Container agnhost-container ready: true, restart count 0
Aug 28 02:55:10.333: INFO: hostexec-node1-xpdzl started at 2021-08-28 02:54:30 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.333: INFO: Container agnhost-container ready: true, restart count 0
Aug 28 02:55:10.333: INFO: kube-multus-ds-amd64-nn7bl started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.333: INFO: Container kube-multus ready: true, restart count 1
Aug 28 02:55:10.333: INFO: kubernetes-metrics-scraper-678c97765c-gtp5x started at 2021-08-27 20:49:21 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.333: INFO: Container kubernetes-metrics-scraper ready: true, restart count 1
Aug 28 02:55:10.333: INFO: cmk-init-discover-node1-spg26 started at 2021-08-27 20:57:37 +0000 UTC (0+3 container statuses recorded)
Aug 28 02:55:10.333: INFO: Container discover ready: false, restart count 0
Aug 28 02:55:10.333: INFO: Container init ready: false, restart count 0
Aug 28 02:55:10.333: INFO: Container install ready: false, restart count 0
Aug 28 02:55:10.333: INFO: nginx-proxy-node1 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.333: INFO: Container nginx-proxy ready: true, restart count 2
Aug 28 02:55:10.333: INFO: hostexec-node1-c46rs started at 2021-08-28 02:54:12 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.333: INFO: Container agnhost-container ready: true, restart count 0
Aug 28 02:55:10.333: INFO: pod-6eb4df2f-adf6-4bdb-9816-33e536823367 started at 2021-08-28 02:53:47 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.333: INFO: Container write-pod ready: false, restart count 0
Aug 28 02:55:10.333: INFO: pod-ffe0bbc2-5867-47ee-b864-cadad4e6110e started at 2021-08-28 02:53:51 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.333: INFO: Container write-pod ready: false, restart count 0
Aug 28 02:55:10.333: INFO: pod-1374eabb-18d0-4da0-8f6d-935c413d0552 started at 2021-08-28 02:54:37 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.333: INFO: Container write-pod ready: false, restart count 0
Aug 28 02:55:10.333: INFO: pod-223d4e9c-285a-4093-9b1f-051e34416c29 started at 2021-08-28 02:54:22 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.333: INFO: Container write-pod ready: false, restart count 0
Aug 28 02:55:10.333: INFO: pod-c6da0a4f-8616-48a3-888b-55d90c658625 started at 2021-08-28 02:54:40 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.333: INFO: Container write-pod ready: false, restart count 0
W0828 02:55:10.345278 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled.
Aug 28 02:55:10.413: INFO:
Latency metrics for node node1
Aug 28 02:55:10.413: INFO:
Logging node info for node node2
Aug 28 02:55:10.416: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 eb7da7c0-513f-4072-a078-ad3d24f88114 133966 0 2021-08-27 20:48:09 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.36.2.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-7451":"csi-mock-csi-mock-volumes-7451","csi-mock-csi-mock-volumes-8521":"csi-mock-csi-mock-volumes-8521"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"9a:88:9c:d1:39:68"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-08-27 20:48:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-08-27 20:55:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-08-27 20:58:03 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-08-28 02:45:47 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-08-28 02:47:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubelet Update v1 2021-08-28 02:49:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {<nil>} 10 DecimalSI},example.com/fakecpu: {{1 3} {<nil>} 1k DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269628928 0} {<nil>} 196552372Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {<nil>} 10 DecimalSI},example.com/fakecpu: {{1 3} {<nil>} 1k DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884628480 0} {<nil>} 174692020Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:50:36 +0000 UTC,LastTransitionTime:2021-08-27 20:50:36 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 02:55:01 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 02:55:01 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 02:55:01 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 02:55:01 +0000 UTC,LastTransitionTime:2021-08-27 20:48:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:0ccfc2a4a9b7400c9ca53b5de0ca4970,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:26b02690-0814-4c92-9f6d-d315df796ce6,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:331c6faa8b0d5ec72cf105e87d35df0a2f2baeec3d6217a51faa73f9460f937f localhost:30500/barometer-collectd:stable],SizeBytes:1238704157,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[localhost:30500/cmk@sha256:fd1487b0c07556a087eff669e70c501a704720dcd53ff75183593de6720585f2 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:d7300ccf7ff3e9cea2111d275143b8050618bbc1d1ffe41f46286b1696261243 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44393508,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:ae72171f047a37ee5423e0692df7429830919af16e9d668ab0c80b723863d102 localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:cd98d1edca8e5e2e3ea42cbc463812483e5d069d10f0974ca9d484b5a7bd68db localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[aquasec/kube-bench@sha256:3544f6662feb73d36fdba35b17652e2fd73aae45bd4b60e76d7ab928220b3cc6 aquasec/kube-bench:0.3.1],SizeBytes:19301876,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Aug 28 02:55:10.417: INFO:
Logging kubelet events for node node2
Aug 28 02:55:10.419: INFO:
Logging pods the kubelet thinks is on node node2
Aug 28 02:55:10.429: INFO: kube-flannel-t9qv4 started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded)
Aug 28 02:55:10.429: INFO: Init container install-cni ready: true, restart count 0
Aug 28 02:55:10.429: INFO: Container kube-flannel ready: true, restart count 2
Aug 28 02:55:10.429: INFO: cmk-init-discover-node2-l9qjd started at 2021-08-27 20:57:57 +0000 UTC (0+3 container statuses recorded)
Aug 28 02:55:10.429: INFO: Container discover ready: false, restart count 0
Aug 28 02:55:10.429: INFO: Container init ready: false, restart count 0
Aug 28 02:55:10.429: INFO: Container install ready: false, restart count 0
Aug 28 02:55:10.429: INFO: node-feature-discovery-worker-54lfh started at 2021-08-27 20:55:06 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.429: INFO: Container nfd-worker ready: true, restart count 0
Aug 28 02:55:10.429: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4f962 started at 2021-08-27 20:55:51 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.429: INFO: Container kube-sriovdp ready: true, restart count 0
Aug 28 02:55:10.429: INFO: nginx-proxy-node2 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.429: INFO: Container nginx-proxy ready: true, restart count 1
Aug 28 02:55:10.429: INFO: kube-proxy-r4q4t started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.429: INFO: Container kube-proxy ready: true, restart count 2
Aug 28 02:55:10.429: INFO: kube-multus-ds-amd64-tfffk started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.429: INFO: Container kube-multus ready: true, restart count 1
Aug 28 02:55:10.429: INFO: node-exporter-p6h5h started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded)
Aug 28 02:55:10.429: INFO: Container kube-rbac-proxy ready: true, restart count 0
Aug 28 02:55:10.429: INFO: Container node-exporter ready: true, restart count 0
Aug 28 02:55:10.429: INFO: cmk-fzjgr started at 2021-08-27 20:58:20 +0000 UTC (0+2 container statuses recorded)
Aug 28 02:55:10.429: INFO: Container nodereport ready: true, restart count 0
Aug 28 02:55:10.429: INFO: Container reconcile ready: true, restart count 0
Aug 28 02:55:10.429: INFO: glusterdynamic-provisioner-6gkg9 started at 2021-08-28 02:52:54 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.429: INFO: Container glusterdynamic-provisioner ready: false, restart count 0
Aug 28 02:55:10.429: INFO: collectd-64dp2 started at 2021-08-27 21:04:15 +0000 UTC (0+3 container statuses recorded)
Aug 28 02:55:10.429: INFO: Container collectd ready: true, restart count 0
Aug 28 02:55:10.429: INFO: Container collectd-exporter ready: true, restart count 0
Aug 28 02:55:10.429: INFO: Container rbac-proxy ready: true, restart count 0
Aug 28 02:55:10.429: INFO: cmk-webhook-6c9d5f8578-ndbx2 started at 2021-08-27 20:58:20 +0000 UTC (0+1 container statuses recorded)
Aug 28 02:55:10.429: INFO: Container cmk-webhook ready: true, restart count 0
Aug 28 02:55:10.430: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-v99df started at 2021-08-27 21:02:08 +0000 UTC (0+2 container statuses recorded)
Aug 28 02:55:10.430: INFO: Container tas-controller ready: true, restart count 0
Aug 28 02:55:10.430: INFO: Container tas-extender ready: true, restart count 0
W0828 02:55:10.443308 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled.
Aug 28 02:55:10.485: INFO:
Latency metrics for node node2
Aug 28 02:55:10.485: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
STEP: Destroying namespace "persistent-local-volumes-test-96" for this suite.
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:263
Aug 28 03:01:23.010: Unexpected error:
<*errors.errorString | 0xc003d8e590>: {
s: "pod \"pod-7125b7ef-99cb-47a1-9ead-e6741d8c15e7\" is not Running: timed out waiting for the condition",
}
pod "pod-7125b7ef-99cb-47a1-9ead-e6741d8c15e7" is not Running: timed out waiting for the condition
occurred
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:793
[BeforeEach] [sig-storage] PersistentVolumes-local
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174
STEP: Creating a kubernetes client
Aug 28 02:59:04.356: INFO: >>> kubeConfig: /root/.kube/config
STEP: Building a namespace api object, basename persistent-local-volumes-test
STEP: Waiting for a default service account to be provisioned in namespace
[BeforeEach] [sig-storage] PersistentVolumes-local
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155
[BeforeEach] [Volume type: blockfswithformat]
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191
STEP: Initializing test volumes
STEP: Creating block device on node "node1" using path "/tmp/local-volume-test-4292005a-bd36-4af4-9cd1-f3cd851e9862"
Aug 28 02:59:08.402: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-4292005a-bd36-4af4-9cd1-f3cd851e9862 && dd if=/dev/zero of=/tmp/local-volume-test-4292005a-bd36-4af4-9cd1-f3cd851e9862/file bs=4096 count=5120 && losetup -f /tmp/local-volume-test-4292005a-bd36-4af4-9cd1-f3cd851e9862/file] Namespace:persistent-local-volumes-test-9574 PodName:hostexec-node1-mw42b ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 02:59:08.402: INFO: >>> kubeConfig: /root/.kube/config
Aug 28 02:59:08.550: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c E2E_LOOP_DEV=$(losetup | grep /tmp/local-volume-test-4292005a-bd36-4af4-9cd1-f3cd851e9862/file | awk '{ print $1 }') 2>&1 > /dev/null && echo ${E2E_LOOP_DEV}] Namespace:persistent-local-volumes-test-9574 PodName:hostexec-node1-mw42b ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 02:59:08.550: INFO: >>> kubeConfig: /root/.kube/config
Aug 28 02:59:08.669: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkfs -t ext4 /dev/loop2 && mount -t ext4 /dev/loop2 /tmp/local-volume-test-4292005a-bd36-4af4-9cd1-f3cd851e9862 && chmod o+rwx /tmp/local-volume-test-4292005a-bd36-4af4-9cd1-f3cd851e9862] Namespace:persistent-local-volumes-test-9574 PodName:hostexec-node1-mw42b ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 02:59:08.669: INFO: >>> kubeConfig: /root/.kube/config
STEP: Creating local PVCs and PVs
Aug 28 02:59:08.957: INFO: Creating a PV followed by a PVC
Aug 28 02:59:08.964: INFO: Waiting for PV local-pv8rs8r to bind to PVC pvc-lkx9x
Aug 28 02:59:08.964: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-lkx9x] to have phase Bound
Aug 28 02:59:08.966: INFO: PersistentVolumeClaim pvc-lkx9x found but phase is Pending instead of Bound.
Aug 28 02:59:10.969: INFO: PersistentVolumeClaim pvc-lkx9x found but phase is Pending instead of Bound.
Aug 28 02:59:12.971: INFO: PersistentVolumeClaim pvc-lkx9x found but phase is Pending instead of Bound.
Aug 28 02:59:14.974: INFO: PersistentVolumeClaim pvc-lkx9x found but phase is Pending instead of Bound.
Aug 28 02:59:16.977: INFO: PersistentVolumeClaim pvc-lkx9x found but phase is Pending instead of Bound.
Aug 28 02:59:18.980: INFO: PersistentVolumeClaim pvc-lkx9x found but phase is Pending instead of Bound.
Aug 28 02:59:20.983: INFO: PersistentVolumeClaim pvc-lkx9x found but phase is Pending instead of Bound.
Aug 28 02:59:22.986: INFO: PersistentVolumeClaim pvc-lkx9x found and phase=Bound (14.022414855s)
Aug 28 02:59:22.986: INFO: Waiting up to 3m0s for PersistentVolume local-pv8rs8r to have phase Bound
Aug 28 02:59:22.988: INFO: PersistentVolume local-pv8rs8r found and phase=Bound (1.930557ms)
[BeforeEach] Set fsGroup for local volume
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:257
[It] should set fsGroup for one pod [Slow]
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:263
STEP: Checking fsGroup is set
STEP: Creating a pod
Aug 28 03:01:23.010: FAIL: Unexpected error:
<*errors.errorString | 0xc003d8e590>: {
s: "pod \"pod-7125b7ef-99cb-47a1-9ead-e6741d8c15e7\" is not Running: timed out waiting for the condition",
}
pod "pod-7125b7ef-99cb-47a1-9ead-e6741d8c15e7" is not Running: timed out waiting for the condition
occurred
Full Stack Trace
k8s.io/kubernetes/test/e2e/storage.createPodWithFsGroupTest(0xc000e77300, 0xc003d7fe90, 0x4d2, 0x4d2, 0x0)
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:793 +0xb7
k8s.io/kubernetes/test/e2e/storage.glob..func20.2.6.2()
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:265 +0x8d
k8s.io/kubernetes/test/e2e.RunE2ETests(0xc0038dcc00)
_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345
k8s.io/kubernetes/test/e2e.TestE2E(0xc0038dcc00)
_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b
testing.tRunner(0xc0038dcc00, 0x4dec428)
/usr/local/go/src/testing/testing.go:1123 +0xef
created by testing.(*T).Run
/usr/local/go/src/testing/testing.go:1168 +0x2b3
[AfterEach] [Volume type: blockfswithformat]
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200
STEP: Cleaning up PVC and PV
Aug 28 03:01:23.012: INFO: Deleting PersistentVolumeClaim "pvc-lkx9x"
Aug 28 03:01:23.017: INFO: Deleting PersistentVolume "local-pv8rs8r"
Aug 28 03:01:23.021: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c umount /tmp/local-volume-test-4292005a-bd36-4af4-9cd1-f3cd851e9862] Namespace:persistent-local-volumes-test-9574 PodName:hostexec-node1-mw42b ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 03:01:23.021: INFO: >>> kubeConfig: /root/.kube/config
Aug 28 03:01:23.277: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c E2E_LOOP_DEV=$(losetup | grep /tmp/local-volume-test-4292005a-bd36-4af4-9cd1-f3cd851e9862/file | awk '{ print $1 }') 2>&1 > /dev/null && echo ${E2E_LOOP_DEV}] Namespace:persistent-local-volumes-test-9574 PodName:hostexec-node1-mw42b ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 03:01:23.278: INFO: >>> kubeConfig: /root/.kube/config
STEP: Tear down block device "/dev/loop2" on node "node1" at path /tmp/local-volume-test-4292005a-bd36-4af4-9cd1-f3cd851e9862/file
Aug 28 03:01:23.404: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c losetup -d /dev/loop2] Namespace:persistent-local-volumes-test-9574 PodName:hostexec-node1-mw42b ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 03:01:23.404: INFO: >>> kubeConfig: /root/.kube/config
STEP: Removing the test directory /tmp/local-volume-test-4292005a-bd36-4af4-9cd1-f3cd851e9862
Aug 28 03:01:23.532: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-4292005a-bd36-4af4-9cd1-f3cd851e9862] Namespace:persistent-local-volumes-test-9574 PodName:hostexec-node1-mw42b ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 03:01:23.532: INFO: >>> kubeConfig: /root/.kube/config
[AfterEach] [sig-storage] PersistentVolumes-local
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175
STEP: Collecting events from namespace "persistent-local-volumes-test-9574".
STEP: Found 15 events.
Aug 28 03:01:23.670: INFO: At 2021-08-28 02:59:04 +0000 UTC - event for hostexec-node1-mw42b: {default-scheduler } Scheduled: Successfully assigned persistent-local-volumes-test-9574/hostexec-node1-mw42b to node1
Aug 28 03:01:23.670: INFO: At 2021-08-28 02:59:04 +0000 UTC - event for hostexec-node1-mw42b: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20"
Aug 28 03:01:23.670: INFO: At 2021-08-28 02:59:05 +0000 UTC - event for hostexec-node1-mw42b: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 506.691556ms
Aug 28 03:01:23.670: INFO: At 2021-08-28 02:59:05 +0000 UTC - event for hostexec-node1-mw42b: {kubelet node1} Created: Created container agnhost-container
Aug 28 03:01:23.670: INFO: At 2021-08-28 02:59:05 +0000 UTC - event for hostexec-node1-mw42b: {kubelet node1} Started: Started container agnhost-container
Aug 28 03:01:23.670: INFO: At 2021-08-28 02:59:08 +0000 UTC - event for pvc-lkx9x: {persistentvolume-controller } ProvisioningFailed: no volume plugin matched name: kubernetes.io/no-provisioner
Aug 28 03:01:23.670: INFO: At 2021-08-28 02:59:23 +0000 UTC - event for pod-7125b7ef-99cb-47a1-9ead-e6741d8c15e7: {default-scheduler } Scheduled: Successfully assigned persistent-local-volumes-test-9574/pod-7125b7ef-99cb-47a1-9ead-e6741d8c15e7 to node1
Aug 28 03:01:23.670: INFO: At 2021-08-28 02:59:29 +0000 UTC - event for pod-7125b7ef-99cb-47a1-9ead-e6741d8c15e7: {multus } AddedInterface: Add eth0 [10.244.3.186/24]
Aug 28 03:01:23.670: INFO: At 2021-08-28 02:59:29 +0000 UTC - event for pod-7125b7ef-99cb-47a1-9ead-e6741d8c15e7: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29"
Aug 28 03:01:23.670: INFO: At 2021-08-28 02:59:30 +0000 UTC - event for pod-7125b7ef-99cb-47a1-9ead-e6741d8c15e7: {kubelet node1} Failed: Error: ErrImagePull
Aug 28 03:01:23.670: INFO: At 2021-08-28 02:59:30 +0000 UTC - event for pod-7125b7ef-99cb-47a1-9ead-e6741d8c15e7: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit
Aug 28 03:01:23.670: INFO: At 2021-08-28 02:59:31 +0000 UTC - event for pod-7125b7ef-99cb-47a1-9ead-e6741d8c15e7: {kubelet node1} SandboxChanged: Pod sandbox changed, it will be killed and re-created.
Aug 28 03:01:23.670: INFO: At 2021-08-28 02:59:33 +0000 UTC - event for pod-7125b7ef-99cb-47a1-9ead-e6741d8c15e7: {multus } AddedInterface: Add eth0 [10.244.3.187/24]
Aug 28 03:01:23.670: INFO: At 2021-08-28 02:59:33 +0000 UTC - event for pod-7125b7ef-99cb-47a1-9ead-e6741d8c15e7: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29"
Aug 28 03:01:23.670: INFO: At 2021-08-28 02:59:33 +0000 UTC - event for pod-7125b7ef-99cb-47a1-9ead-e6741d8c15e7: {kubelet node1} Failed: Error: ImagePullBackOff
Aug 28 03:01:23.674: INFO: POD NODE PHASE GRACE CONDITIONS
Aug 28 03:01:23.674: INFO: hostexec-node1-mw42b node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 02:59:04 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 02:59:06 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 02:59:06 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 02:59:04 +0000 UTC }]
Aug 28 03:01:23.674: INFO: pod-7125b7ef-99cb-47a1-9ead-e6741d8c15e7 node1 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 02:59:23 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-08-28 02:59:23 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-08-28 02:59:23 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 02:59:23 +0000 UTC }]
Aug 28 03:01:23.674: INFO:
Aug 28 03:01:23.678: INFO:
Logging node info for node master1
Aug 28 03:01:23.680: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 3af53387-5aee-42c1-b0e7-644cf9161d48 139946 0 2021-08-27 20:46:13 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"92:f8:b6:72:e4:be"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-08-27 20:46:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-08-27 20:46:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {kube-controller-manager Update v1 2021-08-27 20:46:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:50:56 +0000 UTC,LastTransitionTime:2021-08-27 20:50:56 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 03:01:15 +0000 UTC,LastTransitionTime:2021-08-27 20:46:13 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 03:01:15 +0000 UTC,LastTransitionTime:2021-08-27 20:46:13 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 03:01:15 +0000 UTC,LastTransitionTime:2021-08-27 20:46:13 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 03:01:15 +0000 UTC,LastTransitionTime:2021-08-27 20:50:30 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:5d584023135a46ecb77596bf48ed7f2f,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:587cafa0-6de3-49f8-906e-06315a8ff104,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[sirot/netperf-latest@sha256:23929b922bb077cb341450fc1c90ae42e6f75da5d7ce61cd1880b08560a5ff85 sirot/netperf-latest:latest],SizeBytes:282025213,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:ae72171f047a37ee5423e0692df7429830919af16e9d668ab0c80b723863d102 tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:cd98d1edca8e5e2e3ea42cbc463812483e5d069d10f0974ca9d484b5a7bd68db tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[aquasec/kube-bench@sha256:3544f6662feb73d36fdba35b17652e2fd73aae45bd4b60e76d7ab928220b3cc6 aquasec/kube-bench:0.3.1],SizeBytes:19301876,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Aug 28 03:01:23.680: INFO:
Logging kubelet events for node master1
Aug 28 03:01:23.682: INFO:
Logging pods the kubelet thinks is on node master1
Aug 28 03:01:23.691: INFO: kube-apiserver-master1 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.691: INFO: Container kube-apiserver ready: true, restart count 0
Aug 28 03:01:23.691: INFO: kube-controller-manager-master1 started at 2021-08-27 20:54:27 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.691: INFO: Container kube-controller-manager ready: true, restart count 3
Aug 28 03:01:23.691: INFO: kube-proxy-rb5p6 started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.691: INFO: Container kube-proxy ready: true, restart count 1
Aug 28 03:01:23.691: INFO: coredns-7677f9bb54-dwtp5 started at 2021-08-27 20:49:16 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.691: INFO: Container coredns ready: true, restart count 1
Aug 28 03:01:23.691: INFO: node-exporter-z2ngr started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded)
Aug 28 03:01:23.691: INFO: Container kube-rbac-proxy ready: true, restart count 0
Aug 28 03:01:23.691: INFO: Container node-exporter ready: true, restart count 0
Aug 28 03:01:23.691: INFO: kube-scheduler-master1 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.691: INFO: Container kube-scheduler ready: true, restart count 0
Aug 28 03:01:23.691: INFO: kube-flannel-pp7vp started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded)
Aug 28 03:01:23.691: INFO: Init container install-cni ready: true, restart count 0
Aug 28 03:01:23.691: INFO: Container kube-flannel ready: true, restart count 3
Aug 28 03:01:23.691: INFO: kube-multus-ds-amd64-sfr9k started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.691: INFO: Container kube-multus ready: true, restart count 1
Aug 28 03:01:23.691: INFO: docker-registry-docker-registry-56cbc7bc58-cthtt started at 2021-08-27 20:51:47 +0000 UTC (0+2 container statuses recorded)
Aug 28 03:01:23.691: INFO: Container docker-registry ready: true, restart count 0
Aug 28 03:01:23.691: INFO: Container nginx ready: true, restart count 0
W0828 03:01:23.702582 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled.
Aug 28 03:01:23.730: INFO:
Latency metrics for node master1
Aug 28 03:01:23.730: INFO:
Logging node info for node master2
Aug 28 03:01:23.732: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 666e473f-d9e6-4c06-8b56-06474d788f70 139938 0 2021-08-27 20:46:51 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"42:86:ff:30:bd:4d"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-08-27 20:46:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {kubelet Update v1 2021-08-27 20:46:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-08-27 20:48:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-08-27 20:55:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:51:53 +0000 UTC,LastTransitionTime:2021-08-27 20:51:53 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 03:01:13 +0000 UTC,LastTransitionTime:2021-08-27 20:46:51 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 03:01:13 +0000 UTC,LastTransitionTime:2021-08-27 20:46:51 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 03:01:13 +0000 UTC,LastTransitionTime:2021-08-27 20:46:51 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 03:01:13 +0000 UTC,LastTransitionTime:2021-08-27 20:48:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:2835065974a64998811b9acd85de209b,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:309b9155-1e2a-4ebd-900f-bba5abfc3a5d,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[sirot/netperf-latest@sha256:23929b922bb077cb341450fc1c90ae42e6f75da5d7ce61cd1880b08560a5ff85 sirot/netperf-latest:latest],SizeBytes:282025213,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Aug 28 03:01:23.733: INFO:
Logging kubelet events for node master2
Aug 28 03:01:23.735: INFO:
Logging pods the kubelet thinks is on node master2
Aug 28 03:01:23.743: INFO: node-exporter-96jk5 started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded)
Aug 28 03:01:23.743: INFO: Container kube-rbac-proxy ready: true, restart count 0
Aug 28 03:01:23.743: INFO: Container node-exporter ready: true, restart count 0
Aug 28 03:01:23.743: INFO: kube-controller-manager-master2 started at 2021-08-27 20:51:01 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.743: INFO: Container kube-controller-manager ready: true, restart count 2
Aug 28 03:01:23.743: INFO: kube-flannel-4znnq started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded)
Aug 28 03:01:23.743: INFO: Init container install-cni ready: true, restart count 2
Aug 28 03:01:23.743: INFO: Container kube-flannel ready: true, restart count 2
Aug 28 03:01:23.743: INFO: kube-multus-ds-amd64-4mgbk started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.743: INFO: Container kube-multus ready: true, restart count 1
Aug 28 03:01:23.743: INFO: prometheus-operator-5bb8cb9d8f-whr5p started at 2021-08-27 20:59:06 +0000 UTC (0+2 container statuses recorded)
Aug 28 03:01:23.743: INFO: Container kube-rbac-proxy ready: true, restart count 0
Aug 28 03:01:23.743: INFO: Container prometheus-operator ready: true, restart count 0
Aug 28 03:01:23.743: INFO: kube-apiserver-master2 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.743: INFO: Container kube-apiserver ready: true, restart count 0
Aug 28 03:01:23.743: INFO: kube-scheduler-master2 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.743: INFO: Container kube-scheduler ready: true, restart count 2
Aug 28 03:01:23.743: INFO: kube-proxy-b4mn9 started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.743: INFO: Container kube-proxy ready: true, restart count 1
Aug 28 03:01:23.743: INFO: node-feature-discovery-controller-5bf5c49849-zr9zd started at 2021-08-27 20:55:09 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.743: INFO: Container nfd-controller ready: true, restart count 0
W0828 03:01:23.754703 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled.
Aug 28 03:01:23.782: INFO:
Latency metrics for node master2
Aug 28 03:01:23.782: INFO:
Logging node info for node master3
Aug 28 03:01:23.784: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 620b6dba-f2c5-46e9-b2ff-d2f4197167d0 139990 0 2021-08-27 20:47:02 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ce:de:ea:c3:40:4f"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-08-27 20:47:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-08-27 20:47:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-08-27 20:48:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:51:44 +0000 UTC,LastTransitionTime:2021-08-27 20:51:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 03:01:21 +0000 UTC,LastTransitionTime:2021-08-27 20:47:02 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 03:01:21 +0000 UTC,LastTransitionTime:2021-08-27 20:47:02 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 03:01:21 +0000 UTC,LastTransitionTime:2021-08-27 20:47:02 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 03:01:21 +0000 UTC,LastTransitionTime:2021-08-27 20:48:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:37decbffe0e84048b5801289ad3be5bf,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:c96fe8b9-1ce0-44cd-935a-b58987e26570,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496891,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Aug 28 03:01:23.785: INFO:
Logging kubelet events for node master3
Aug 28 03:01:23.787: INFO:
Logging pods the kubelet thinks is on node master3
Aug 28 03:01:23.795: INFO: coredns-7677f9bb54-rxplt started at 2021-08-27 20:49:21 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.795: INFO: Container coredns ready: true, restart count 1
Aug 28 03:01:23.795: INFO: kube-controller-manager-master3 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.795: INFO: Container kube-controller-manager ready: true, restart count 2
Aug 28 03:01:23.795: INFO: kube-proxy-8sxhm started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.795: INFO: Container kube-proxy ready: true, restart count 1
Aug 28 03:01:23.795: INFO: kube-flannel-fkz5d started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded)
Aug 28 03:01:23.795: INFO: Init container install-cni ready: true, restart count 0
Aug 28 03:01:23.795: INFO: Container kube-flannel ready: true, restart count 1
Aug 28 03:01:23.795: INFO: dns-autoscaler-5b7b5c9b6f-54xch started at 2021-08-27 20:49:19 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.795: INFO: Container autoscaler ready: true, restart count 1
Aug 28 03:01:23.795: INFO: kube-apiserver-master3 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.795: INFO: Container kube-apiserver ready: true, restart count 0
Aug 28 03:01:23.795: INFO: kube-scheduler-master3 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.795: INFO: Container kube-scheduler ready: true, restart count 2
Aug 28 03:01:23.795: INFO: kube-multus-ds-amd64-wwcgv started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.795: INFO: Container kube-multus ready: true, restart count 1
Aug 28 03:01:23.795: INFO: node-exporter-d4m7q started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded)
Aug 28 03:01:23.795: INFO: Container kube-rbac-proxy ready: true, restart count 0
Aug 28 03:01:23.795: INFO: Container node-exporter ready: true, restart count 0
W0828 03:01:23.808921 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled.
Aug 28 03:01:23.839: INFO:
Latency metrics for node master3
Aug 28 03:01:23.839: INFO:
Logging node info for node node1
Aug 28 03:01:23.841: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 e7a2481e-32db-4c83-bd9f-4a0687258e7a 139967 0 2021-08-27 20:48:09 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.36.2.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1382":"csi-mock-csi-mock-volumes-1382","csi-mock-csi-mock-volumes-1844":"csi-mock-csi-mock-volumes-1844","csi-mock-csi-mock-volumes-2731":"csi-mock-csi-mock-volumes-2731","csi-mock-csi-mock-volumes-4084":"csi-mock-csi-mock-volumes-4084","csi-mock-csi-mock-volumes-6046":"csi-mock-csi-mock-volumes-6046","csi-mock-csi-mock-volumes-6248":"csi-mock-csi-mock-volumes-6248","csi-mock-csi-mock-volumes-7866":"csi-mock-csi-mock-volumes-7866","csi-mock-csi-mock-volumes-9165":"csi-mock-csi-mock-volumes-9165","csi-mock-csi-mock-volumes-9410":"csi-mock-csi-mock-volumes-9410"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"56:c7:37:40:51:ca"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-08-27 20:48:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-08-27 20:55:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-08-27 20:57:40 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-08-28 02:45:58 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-08-28 02:58:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubelet Update v1 2021-08-28 02:58:38 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:io.kubernetes.storage.mock/node":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {<nil>} 10 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269628928 0} {<nil>} 196552372Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {<nil>} 10 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884628480 0} {<nil>} 174692020Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:51:49 +0000 UTC,LastTransitionTime:2021-08-27 20:51:49 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 03:01:20 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 03:01:20 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 03:01:20 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 03:01:20 +0000 UTC,LastTransitionTime:2021-08-27 20:48:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:c1e38e80ea114a5f96601202301ce842,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:e769e86d-15c0-442c-a93b-bcc6c33ff1cd,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:331c6faa8b0d5ec72cf105e87d35df0a2f2baeec3d6217a51faa73f9460f937f localhost:30500/barometer-collectd:stable],SizeBytes:1238704157,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:a7cea43d9d2f67c38fbf0407786edbe660ee9072945f7bb272b55fd255e8eaca opnfv/barometer-collectd:stable],SizeBytes:1075746799,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:1003787960,},ContainerImage{Names:[localhost:30500/cmk@sha256:fd1487b0c07556a087eff669e70c501a704720dcd53ff75183593de6720585f2 cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[aquasec/kube-hunter@sha256:2be6820bc1d7e0f57193a9a27d5a3e16b2fd93c53747b03ce8ca48c6fc323781 aquasec/kube-hunter:0.3.1],SizeBytes:347611549,},ContainerImage{Names:[golang@sha256:db2475a1dbb2149508e5db31d7d77a75e6600d54be645f37681f03f2762169ba golang:alpine3.12],SizeBytes:301186719,},ContainerImage{Names:[sirot/netperf-latest@sha256:23929b922bb077cb341450fc1c90ae42e6f75da5d7ce61cd1880b08560a5ff85 sirot/netperf-latest:latest],SizeBytes:282025213,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:d7300ccf7ff3e9cea2111d275143b8050618bbc1d1ffe41f46286b1696261243 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44393508,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/regression-issue-74839-amd64@sha256:3b36bd80b97c532a774e7f6246797b8575d97037982f353476c703ba6686c75c gcr.io/kubernetes-e2e-test-images/regression-issue-74839-amd64:1.0],SizeBytes:19227369,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:de25c7fc6c4f3a27c7f0c2dff454e4671823a34d88abd533f210848d527e0fbb alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Aug 28 03:01:23.842: INFO:
Logging kubelet events for node node1
Aug 28 03:01:23.844: INFO:
Logging pods the kubelet thinks is on node node1
Aug 28 03:01:23.862: INFO: pod-66cfcad5-5dbe-4dea-ba40-62eaa829c8e1 started at 2021-08-28 03:01:07 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.862: INFO: Container write-pod ready: false, restart count 0
Aug 28 03:01:23.862: INFO: hostexec-node1-mjhpb started at <nil> (0+0 container statuses recorded)
Aug 28 03:01:23.862: INFO: kube-flannel-ssxn7 started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded)
Aug 28 03:01:23.862: INFO: Init container install-cni ready: true, restart count 0
Aug 28 03:01:23.862: INFO: Container kube-flannel ready: true, restart count 1
Aug 28 03:01:23.862: INFO: cmk-jw4m6 started at 2021-08-27 20:58:19 +0000 UTC (0+2 container statuses recorded)
Aug 28 03:01:23.862: INFO: Container nodereport ready: true, restart count 0
Aug 28 03:01:23.862: INFO: Container reconcile ready: true, restart count 0
Aug 28 03:01:23.862: INFO: hostexec-node1-mw42b started at 2021-08-28 02:59:04 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.862: INFO: Container agnhost-container ready: true, restart count 0
Aug 28 03:01:23.862: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-9lndx started at 2021-08-27 20:55:51 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.862: INFO: Container kube-sriovdp ready: true, restart count 0
Aug 28 03:01:23.862: INFO: node-exporter-4cvlq started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded)
Aug 28 03:01:23.862: INFO: Container kube-rbac-proxy ready: true, restart count 0
Aug 28 03:01:23.862: INFO: Container node-exporter ready: true, restart count 0
Aug 28 03:01:23.862: INFO: hostexec-node1-ffnrn started at 2021-08-28 02:58:58 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.862: INFO: Container agnhost-container ready: true, restart count 0
Aug 28 03:01:23.862: INFO: pod-fcaf06c2-9c0a-481b-b54e-82598e6c5dd7 started at 2021-08-28 03:00:52 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.862: INFO: Container write-pod ready: false, restart count 0
Aug 28 03:01:23.862: INFO: kubernetes-dashboard-86c6f9df5b-c56fg started at 2021-08-27 20:49:21 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.862: INFO: Container kubernetes-dashboard ready: true, restart count 1
Aug 28 03:01:23.862: INFO: node-feature-discovery-worker-bd9kg started at 2021-08-27 20:55:06 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.862: INFO: Container nfd-worker ready: true, restart count 0
Aug 28 03:01:23.862: INFO: hostexec-node1-ztv9s started at 2021-08-28 02:56:55 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.862: INFO: Container agnhost-container ready: true, restart count 0
Aug 28 03:01:23.862: INFO: pod-7125b7ef-99cb-47a1-9ead-e6741d8c15e7 started at 2021-08-28 02:59:23 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.862: INFO: Container write-pod ready: false, restart count 0
Aug 28 03:01:23.862: INFO: hostexec-node1-kjhl9 started at 2021-08-28 03:01:01 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.862: INFO: Container agnhost-container ready: true, restart count 0
Aug 28 03:01:23.862: INFO: pod-74ff2226-c67e-4e98-af28-85c2ac4557fd started at 2021-08-28 03:01:23 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.862: INFO: Container write-pod ready: false, restart count 0
Aug 28 03:01:23.862: INFO: hostexec-node1-76w99 started at <nil> (0+0 container statuses recorded)
Aug 28 03:01:23.862: INFO: prometheus-k8s-0 started at 2021-08-27 20:59:29 +0000 UTC (0+5 container statuses recorded)
Aug 28 03:01:23.862: INFO: Container custom-metrics-apiserver ready: true, restart count 0
Aug 28 03:01:23.862: INFO: Container grafana ready: true, restart count 0
Aug 28 03:01:23.862: INFO: Container prometheus ready: true, restart count 1
Aug 28 03:01:23.862: INFO: Container prometheus-config-reloader ready: true, restart count 0
Aug 28 03:01:23.862: INFO: Container rules-configmap-reloader ready: true, restart count 0
Aug 28 03:01:23.862: INFO: pod-eeab6ba8-de2a-4c50-a640-5c37170b96e0 started at 2021-08-28 02:59:37 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.862: INFO: Container write-pod ready: false, restart count 0
Aug 28 03:01:23.862: INFO: kube-multus-ds-amd64-nn7bl started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.862: INFO: Container kube-multus ready: true, restart count 1
Aug 28 03:01:23.862: INFO: kubernetes-metrics-scraper-678c97765c-gtp5x started at 2021-08-27 20:49:21 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.862: INFO: Container kubernetes-metrics-scraper ready: true, restart count 1
Aug 28 03:01:23.862: INFO: cmk-init-discover-node1-spg26 started at 2021-08-27 20:57:37 +0000 UTC (0+3 container statuses recorded)
Aug 28 03:01:23.862: INFO: Container discover ready: false, restart count 0
Aug 28 03:01:23.862: INFO: Container init ready: false, restart count 0
Aug 28 03:01:23.862: INFO: Container install ready: false, restart count 0
Aug 28 03:01:23.862: INFO: hostexec-node1-h2p4v started at 2021-08-28 02:59:16 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.862: INFO: Container agnhost-container ready: true, restart count 0
Aug 28 03:01:23.862: INFO: nginx-proxy-node1 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.862: INFO: Container nginx-proxy ready: true, restart count 2
Aug 28 03:01:23.862: INFO: pod-bc85351d-0740-4a93-a4fb-0e46bebdf868 started at 2021-08-28 02:59:22 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.862: INFO: Container write-pod ready: false, restart count 0
Aug 28 03:01:23.862: INFO: hostexec-node1-2rpsd started at 2021-08-28 03:01:06 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.862: INFO: Container agnhost-container ready: true, restart count 0
Aug 28 03:01:23.862: INFO: hostexec-node1-mk2kd started at 2021-08-28 03:00:40 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.862: INFO: Container agnhost-container ready: true, restart count 0
Aug 28 03:01:23.862: INFO: kube-proxy-pb5bl started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.862: INFO: Container kube-proxy ready: true, restart count 2
Aug 28 03:01:23.862: INFO: collectd-ccvwg started at 2021-08-27 21:04:15 +0000 UTC (0+3 container statuses recorded)
Aug 28 03:01:23.862: INFO: Container collectd ready: true, restart count 0
Aug 28 03:01:23.862: INFO: Container collectd-exporter ready: true, restart count 0
Aug 28 03:01:23.862: INFO: Container rbac-proxy ready: true, restart count 0
Aug 28 03:01:23.862: INFO: hostexec-node1-l9jtf started at 2021-08-28 02:59:22 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.862: INFO: Container agnhost-container ready: true, restart count 0
Aug 28 03:01:23.862: INFO: pod-3d1abce1-f9be-43bf-8929-afbdcf56880c started at 2021-08-28 02:59:22 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.862: INFO: Container write-pod ready: false, restart count 0
W0828 03:01:23.875812 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled.
Aug 28 03:01:23.919: INFO:
Latency metrics for node node1
Aug 28 03:01:23.919: INFO:
Logging node info for node node2
Aug 28 03:01:23.923: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 eb7da7c0-513f-4072-a078-ad3d24f88114 140027 0 2021-08-27 20:48:09 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.36.2.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1442":"csi-mock-csi-mock-volumes-1442","csi-mock-csi-mock-volumes-2967":"csi-mock-csi-mock-volumes-2967","csi-mock-csi-mock-volumes-7451":"csi-mock-csi-mock-volumes-7451","csi-mock-csi-mock-volumes-8521":"csi-mock-csi-mock-volumes-8521"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"9a:88:9c:d1:39:68"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-08-27 20:48:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-08-27 20:55:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-08-27 20:58:03 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-08-28 02:45:47 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-08-28 02:57:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubelet Update v1 2021-08-28 02:57:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {<nil>} 10 DecimalSI},example.com/fakecpu: {{1 3} {<nil>} 1k DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269628928 0} {<nil>} 196552372Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {<nil>} 10 DecimalSI},example.com/fakecpu: {{1 3} {<nil>} 1k DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884628480 0} {<nil>} 174692020Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:50:36 +0000 UTC,LastTransitionTime:2021-08-27 20:50:36 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 03:01:23 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 03:01:23 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 03:01:23 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 03:01:23 +0000 UTC,LastTransitionTime:2021-08-27 20:48:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:0ccfc2a4a9b7400c9ca53b5de0ca4970,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:26b02690-0814-4c92-9f6d-d315df796ce6,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:331c6faa8b0d5ec72cf105e87d35df0a2f2baeec3d6217a51faa73f9460f937f localhost:30500/barometer-collectd:stable],SizeBytes:1238704157,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[localhost:30500/cmk@sha256:fd1487b0c07556a087eff669e70c501a704720dcd53ff75183593de6720585f2 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[k8s.gcr.io/sig-storage/nfs-provisioner@sha256:c1bedac8758029948afe060bf8f6ee63ea489b5e08d29745f44fab68ee0d46ca k8s.gcr.io/sig-storage/nfs-provisioner:v2.2.2],SizeBytes:391772778,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:d7300ccf7ff3e9cea2111d275143b8050618bbc1d1ffe41f46286b1696261243 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44393508,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:ae72171f047a37ee5423e0692df7429830919af16e9d668ab0c80b723863d102 localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:cd98d1edca8e5e2e3ea42cbc463812483e5d069d10f0974ca9d484b5a7bd68db localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[aquasec/kube-bench@sha256:3544f6662feb73d36fdba35b17652e2fd73aae45bd4b60e76d7ab928220b3cc6 aquasec/kube-bench:0.3.1],SizeBytes:19301876,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Aug 28 03:01:23.924: INFO:
Logging kubelet events for node node2
Aug 28 03:01:23.926: INFO:
Logging pods the kubelet thinks is on node node2
Aug 28 03:01:23.938: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-v99df started at 2021-08-27 21:02:08 +0000 UTC (0+2 container statuses recorded)
Aug 28 03:01:23.938: INFO: Container tas-controller ready: true, restart count 0
Aug 28 03:01:23.938: INFO: Container tas-extender ready: true, restart count 0
Aug 28 03:01:23.938: INFO: cmk-webhook-6c9d5f8578-ndbx2 started at 2021-08-27 20:58:20 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.938: INFO: Container cmk-webhook ready: true, restart count 0
Aug 28 03:01:23.938: INFO: cmk-init-discover-node2-l9qjd started at 2021-08-27 20:57:57 +0000 UTC (0+3 container statuses recorded)
Aug 28 03:01:23.938: INFO: Container discover ready: false, restart count 0
Aug 28 03:01:23.938: INFO: Container init ready: false, restart count 0
Aug 28 03:01:23.938: INFO: Container install ready: false, restart count 0
Aug 28 03:01:23.938: INFO: kube-flannel-t9qv4 started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded)
Aug 28 03:01:23.938: INFO: Init container install-cni ready: true, restart count 0
Aug 28 03:01:23.938: INFO: Container kube-flannel ready: true, restart count 2
Aug 28 03:01:23.938: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4f962 started at 2021-08-27 20:55:51 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.938: INFO: Container kube-sriovdp ready: true, restart count 0
Aug 28 03:01:23.938: INFO: hostexec-node2-q48ft started at 2021-08-28 02:57:12 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.938: INFO: Container agnhost-container ready: true, restart count 0
Aug 28 03:01:23.938: INFO: node-feature-discovery-worker-54lfh started at 2021-08-27 20:55:06 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.938: INFO: Container nfd-worker ready: true, restart count 0
Aug 28 03:01:23.938: INFO: kube-proxy-r4q4t started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.938: INFO: Container kube-proxy ready: true, restart count 2
Aug 28 03:01:23.938: INFO: kube-multus-ds-amd64-tfffk started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.938: INFO: Container kube-multus ready: true, restart count 1
Aug 28 03:01:23.938: INFO: node-exporter-p6h5h started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded)
Aug 28 03:01:23.938: INFO: Container kube-rbac-proxy ready: true, restart count 0
Aug 28 03:01:23.938: INFO: Container node-exporter ready: true, restart count 0
Aug 28 03:01:23.938: INFO: pod-secrets-92f12714-401d-4089-82b5-6e8014f9ef8e started at 2021-08-28 02:59:07 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.938: INFO: Container creates-volume-test ready: false, restart count 0
Aug 28 03:01:23.938: INFO: nginx-proxy-node2 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.938: INFO: Container nginx-proxy ready: true, restart count 1
Aug 28 03:01:23.938: INFO: pod-projected-configmaps-f2987d84-1578-404d-96ea-c475a81dcba6 started at 2021-08-28 02:57:42 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.938: INFO: Container createcm-volume-test ready: false, restart count 0
Aug 28 03:01:23.938: INFO: cmk-fzjgr started at 2021-08-27 20:58:20 +0000 UTC (0+2 container statuses recorded)
Aug 28 03:01:23.938: INFO: Container nodereport ready: true, restart count 0
Aug 28 03:01:23.938: INFO: Container reconcile ready: true, restart count 0
Aug 28 03:01:23.938: INFO: local-volume-statefulset-1 started at 2021-08-28 02:57:28 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.938: INFO: Container nginx ready: false, restart count 0
Aug 28 03:01:23.938: INFO: local-volume-statefulset-2 started at 2021-08-28 02:57:28 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.938: INFO: Container nginx ready: false, restart count 0
Aug 28 03:01:23.938: INFO: local-volume-statefulset-0 started at 2021-08-28 02:57:28 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:01:23.938: INFO: Container nginx ready: false, restart count 0
Aug 28 03:01:23.938: INFO: collectd-64dp2 started at 2021-08-27 21:04:15 +0000 UTC (0+3 container statuses recorded)
Aug 28 03:01:23.938: INFO: Container collectd ready: true, restart count 0
Aug 28 03:01:23.938: INFO: Container collectd-exporter ready: true, restart count 0
Aug 28 03:01:23.938: INFO: Container rbac-proxy ready: true, restart count 0
W0828 03:01:23.951007 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled.
Aug 28 03:01:23.994: INFO:
Latency metrics for node node2
Aug 28 03:01:23.994: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
STEP: Destroying namespace "persistent-local-volumes-test-9574" for this suite.
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:245
Aug 28 03:03:52.813: Unexpected error:
<*errors.errorString | 0xc002808a20>: {
s: "pod \"pod-9c953050-6df2-4b46-88fa-b80b115a1dd1\" is not Running: timed out waiting for the condition",
}
pod "pod-9c953050-6df2-4b46-88fa-b80b115a1dd1" is not Running: timed out waiting for the condition
occurred
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:727
[BeforeEach] [sig-storage] PersistentVolumes-local
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174
STEP: Creating a kubernetes client
Aug 28 03:01:36.354: INFO: >>> kubeConfig: /root/.kube/config
STEP: Building a namespace api object, basename persistent-local-volumes-test
STEP: Waiting for a default service account to be provisioned in namespace
[BeforeEach] [sig-storage] PersistentVolumes-local
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155
[BeforeEach] [Volume type: block]
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191
STEP: Initializing test volumes
STEP: Creating block device on node "node1" using path "/tmp/local-volume-test-b8705afd-ea90-4673-aa60-1f71321fa9ee"
Aug 28 03:01:38.409: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-b8705afd-ea90-4673-aa60-1f71321fa9ee && dd if=/dev/zero of=/tmp/local-volume-test-b8705afd-ea90-4673-aa60-1f71321fa9ee/file bs=4096 count=5120 && losetup -f /tmp/local-volume-test-b8705afd-ea90-4673-aa60-1f71321fa9ee/file] Namespace:persistent-local-volumes-test-8743 PodName:hostexec-node1-x62tg ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 03:01:38.409: INFO: >>> kubeConfig: /root/.kube/config
Aug 28 03:01:38.631: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c E2E_LOOP_DEV=$(losetup | grep /tmp/local-volume-test-b8705afd-ea90-4673-aa60-1f71321fa9ee/file | awk '{ print $1 }') 2>&1 > /dev/null && echo ${E2E_LOOP_DEV}] Namespace:persistent-local-volumes-test-8743 PodName:hostexec-node1-x62tg ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 03:01:38.631: INFO: >>> kubeConfig: /root/.kube/config
STEP: Creating local PVCs and PVs
Aug 28 03:01:38.747: INFO: Creating a PV followed by a PVC
Aug 28 03:01:38.754: INFO: Waiting for PV local-pvmxd9f to bind to PVC pvc-m5662
Aug 28 03:01:38.754: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-m5662] to have phase Bound
Aug 28 03:01:38.758: INFO: PersistentVolumeClaim pvc-m5662 found but phase is Pending instead of Bound.
Aug 28 03:01:40.761: INFO: PersistentVolumeClaim pvc-m5662 found but phase is Pending instead of Bound.
Aug 28 03:01:42.764: INFO: PersistentVolumeClaim pvc-m5662 found but phase is Pending instead of Bound.
Aug 28 03:01:44.769: INFO: PersistentVolumeClaim pvc-m5662 found but phase is Pending instead of Bound.
Aug 28 03:01:46.774: INFO: PersistentVolumeClaim pvc-m5662 found but phase is Pending instead of Bound.
Aug 28 03:01:48.779: INFO: PersistentVolumeClaim pvc-m5662 found but phase is Pending instead of Bound.
Aug 28 03:01:50.782: INFO: PersistentVolumeClaim pvc-m5662 found but phase is Pending instead of Bound.
Aug 28 03:01:52.785: INFO: PersistentVolumeClaim pvc-m5662 found and phase=Bound (14.031209578s)
Aug 28 03:01:52.785: INFO: Waiting up to 3m0s for PersistentVolume local-pvmxd9f to have phase Bound
Aug 28 03:01:52.788: INFO: PersistentVolume local-pvmxd9f found and phase=Bound (3.047139ms)
[It] should be able to write from pod1 and read from pod2
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:245
STEP: Creating pod1 to write to the PV
STEP: Creating a pod
Aug 28 03:03:52.813: FAIL: Unexpected error:
<*errors.errorString | 0xc002808a20>: {
s: "pod \"pod-9c953050-6df2-4b46-88fa-b80b115a1dd1\" is not Running: timed out waiting for the condition",
}
pod "pod-9c953050-6df2-4b46-88fa-b80b115a1dd1" is not Running: timed out waiting for the condition
occurred
Full Stack Trace
k8s.io/kubernetes/test/e2e/storage.twoPodsReadWriteTest(0xc002b50c60, 0xc002188a80, 0xc0023f9710)
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:727 +0xaf
k8s.io/kubernetes/test/e2e/storage.glob..func20.2.4.1()
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:246 +0x45
k8s.io/kubernetes/test/e2e.RunE2ETests(0xc0038dcc00)
_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345
k8s.io/kubernetes/test/e2e.TestE2E(0xc0038dcc00)
_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b
testing.tRunner(0xc0038dcc00, 0x4dec428)
/usr/local/go/src/testing/testing.go:1123 +0xef
created by testing.(*T).Run
/usr/local/go/src/testing/testing.go:1168 +0x2b3
[AfterEach] [Volume type: block]
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200
STEP: Cleaning up PVC and PV
Aug 28 03:03:52.815: INFO: Deleting PersistentVolumeClaim "pvc-m5662"
Aug 28 03:03:52.819: INFO: Deleting PersistentVolume "local-pvmxd9f"
Aug 28 03:03:52.823: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c E2E_LOOP_DEV=$(losetup | grep /tmp/local-volume-test-b8705afd-ea90-4673-aa60-1f71321fa9ee/file | awk '{ print $1 }') 2>&1 > /dev/null && echo ${E2E_LOOP_DEV}] Namespace:persistent-local-volumes-test-8743 PodName:hostexec-node1-x62tg ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 03:03:52.823: INFO: >>> kubeConfig: /root/.kube/config
STEP: Tear down block device "/dev/loop2" on node "node1" at path /tmp/local-volume-test-b8705afd-ea90-4673-aa60-1f71321fa9ee/file
Aug 28 03:03:53.023: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c losetup -d /dev/loop2] Namespace:persistent-local-volumes-test-8743 PodName:hostexec-node1-x62tg ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 03:03:53.023: INFO: >>> kubeConfig: /root/.kube/config
STEP: Removing the test directory /tmp/local-volume-test-b8705afd-ea90-4673-aa60-1f71321fa9ee
Aug 28 03:03:53.211: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-b8705afd-ea90-4673-aa60-1f71321fa9ee] Namespace:persistent-local-volumes-test-8743 PodName:hostexec-node1-x62tg ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 03:03:53.211: INFO: >>> kubeConfig: /root/.kube/config
[AfterEach] [sig-storage] PersistentVolumes-local
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175
STEP: Collecting events from namespace "persistent-local-volumes-test-8743".
STEP: Found 15 events.
Aug 28 03:03:53.419: INFO: At 2021-08-28 03:01:36 +0000 UTC - event for hostexec-node1-x62tg: {default-scheduler } Scheduled: Successfully assigned persistent-local-volumes-test-8743/hostexec-node1-x62tg to node1
Aug 28 03:03:53.419: INFO: At 2021-08-28 03:01:36 +0000 UTC - event for hostexec-node1-x62tg: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20"
Aug 28 03:03:53.419: INFO: At 2021-08-28 03:01:37 +0000 UTC - event for hostexec-node1-x62tg: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 534.618452ms
Aug 28 03:03:53.419: INFO: At 2021-08-28 03:01:37 +0000 UTC - event for hostexec-node1-x62tg: {kubelet node1} Created: Created container agnhost-container
Aug 28 03:03:53.419: INFO: At 2021-08-28 03:01:37 +0000 UTC - event for hostexec-node1-x62tg: {kubelet node1} Started: Started container agnhost-container
Aug 28 03:03:53.419: INFO: At 2021-08-28 03:01:38 +0000 UTC - event for pvc-m5662: {persistentvolume-controller } ProvisioningFailed: no volume plugin matched name: kubernetes.io/no-provisioner
Aug 28 03:03:53.419: INFO: At 2021-08-28 03:01:52 +0000 UTC - event for pod-9c953050-6df2-4b46-88fa-b80b115a1dd1: {default-scheduler } Scheduled: Successfully assigned persistent-local-volumes-test-8743/pod-9c953050-6df2-4b46-88fa-b80b115a1dd1 to node1
Aug 28 03:03:53.419: INFO: At 2021-08-28 03:01:53 +0000 UTC - event for pod-9c953050-6df2-4b46-88fa-b80b115a1dd1: {kubelet node1} SuccessfulMountVolume: MapVolume.MapPodDevice succeeded for volume "local-pvmxd9f" globalMapPath "/var/lib/kubelet/plugins/kubernetes.io~local-volume/volumeDevices/local-pvmxd9f"
Aug 28 03:03:53.419: INFO: At 2021-08-28 03:01:53 +0000 UTC - event for pod-9c953050-6df2-4b46-88fa-b80b115a1dd1: {kubelet node1} SuccessfulMountVolume: MapVolume.MapPodDevice succeeded for volume "local-pvmxd9f" volumeMapPath "/var/lib/kubelet/pods/066ac6bc-cf48-4714-925b-4587908ccd82/volumeDevices/kubernetes.io~local-volume"
Aug 28 03:03:53.419: INFO: At 2021-08-28 03:01:54 +0000 UTC - event for pod-9c953050-6df2-4b46-88fa-b80b115a1dd1: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29"
Aug 28 03:03:53.419: INFO: At 2021-08-28 03:01:54 +0000 UTC - event for pod-9c953050-6df2-4b46-88fa-b80b115a1dd1: {multus } AddedInterface: Add eth0 [10.244.3.196/24]
Aug 28 03:03:53.419: INFO: At 2021-08-28 03:01:55 +0000 UTC - event for pod-9c953050-6df2-4b46-88fa-b80b115a1dd1: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit
Aug 28 03:03:53.419: INFO: At 2021-08-28 03:01:55 +0000 UTC - event for pod-9c953050-6df2-4b46-88fa-b80b115a1dd1: {kubelet node1} Failed: Error: ErrImagePull
Aug 28 03:03:53.419: INFO: At 2021-08-28 03:01:56 +0000 UTC - event for pod-9c953050-6df2-4b46-88fa-b80b115a1dd1: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29"
Aug 28 03:03:53.419: INFO: At 2021-08-28 03:01:56 +0000 UTC - event for pod-9c953050-6df2-4b46-88fa-b80b115a1dd1: {kubelet node1} Failed: Error: ImagePullBackOff
Aug 28 03:03:53.422: INFO: POD NODE PHASE GRACE CONDITIONS
Aug 28 03:03:53.422: INFO: hostexec-node1-x62tg node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 03:01:36 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 03:01:38 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 03:01:38 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 03:01:36 +0000 UTC }]
Aug 28 03:03:53.422: INFO: pod-9c953050-6df2-4b46-88fa-b80b115a1dd1 node1 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 03:01:52 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-08-28 03:01:52 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-08-28 03:01:52 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 03:01:52 +0000 UTC }]
Aug 28 03:03:53.422: INFO:
Aug 28 03:03:53.427: INFO:
Logging node info for node master1
Aug 28 03:03:53.430: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 3af53387-5aee-42c1-b0e7-644cf9161d48 142492 0 2021-08-27 20:46:13 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"92:f8:b6:72:e4:be"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-08-27 20:46:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-08-27 20:46:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {kube-controller-manager Update v1 2021-08-27 20:46:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:50:56 +0000 UTC,LastTransitionTime:2021-08-27 20:50:56 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 03:03:45 +0000 UTC,LastTransitionTime:2021-08-27 20:46:13 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 03:03:45 +0000 UTC,LastTransitionTime:2021-08-27 20:46:13 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 03:03:45 +0000 UTC,LastTransitionTime:2021-08-27 20:46:13 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 03:03:45 +0000 UTC,LastTransitionTime:2021-08-27 20:50:30 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:5d584023135a46ecb77596bf48ed7f2f,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:587cafa0-6de3-49f8-906e-06315a8ff104,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[sirot/netperf-latest@sha256:23929b922bb077cb341450fc1c90ae42e6f75da5d7ce61cd1880b08560a5ff85 sirot/netperf-latest:latest],SizeBytes:282025213,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:ae72171f047a37ee5423e0692df7429830919af16e9d668ab0c80b723863d102 tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:cd98d1edca8e5e2e3ea42cbc463812483e5d069d10f0974ca9d484b5a7bd68db tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[aquasec/kube-bench@sha256:3544f6662feb73d36fdba35b17652e2fd73aae45bd4b60e76d7ab928220b3cc6 aquasec/kube-bench:0.3.1],SizeBytes:19301876,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Aug 28 03:03:53.431: INFO:
Logging kubelet events for node master1
Aug 28 03:03:53.433: INFO:
Logging pods the kubelet thinks is on node master1
Aug 28 03:03:53.442: INFO: kube-scheduler-master1 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.442: INFO: Container kube-scheduler ready: true, restart count 0
Aug 28 03:03:53.442: INFO: kube-flannel-pp7vp started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded)
Aug 28 03:03:53.442: INFO: Init container install-cni ready: true, restart count 0
Aug 28 03:03:53.442: INFO: Container kube-flannel ready: true, restart count 3
Aug 28 03:03:53.442: INFO: kube-multus-ds-amd64-sfr9k started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.442: INFO: Container kube-multus ready: true, restart count 1
Aug 28 03:03:53.442: INFO: docker-registry-docker-registry-56cbc7bc58-cthtt started at 2021-08-27 20:51:47 +0000 UTC (0+2 container statuses recorded)
Aug 28 03:03:53.442: INFO: Container docker-registry ready: true, restart count 0
Aug 28 03:03:53.442: INFO: Container nginx ready: true, restart count 0
Aug 28 03:03:53.442: INFO: node-exporter-z2ngr started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded)
Aug 28 03:03:53.442: INFO: Container kube-rbac-proxy ready: true, restart count 0
Aug 28 03:03:53.442: INFO: Container node-exporter ready: true, restart count 0
Aug 28 03:03:53.442: INFO: kube-apiserver-master1 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.442: INFO: Container kube-apiserver ready: true, restart count 0
Aug 28 03:03:53.442: INFO: kube-controller-manager-master1 started at 2021-08-27 20:54:27 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.442: INFO: Container kube-controller-manager ready: true, restart count 3
Aug 28 03:03:53.442: INFO: kube-proxy-rb5p6 started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.442: INFO: Container kube-proxy ready: true, restart count 1
Aug 28 03:03:53.442: INFO: coredns-7677f9bb54-dwtp5 started at 2021-08-27 20:49:16 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.442: INFO: Container coredns ready: true, restart count 1
W0828 03:03:53.455510 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled.
Aug 28 03:03:53.491: INFO:
Latency metrics for node master1
Aug 28 03:03:53.491: INFO:
Logging node info for node master2
Aug 28 03:03:53.494: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 666e473f-d9e6-4c06-8b56-06474d788f70 142473 0 2021-08-27 20:46:51 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"42:86:ff:30:bd:4d"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-08-27 20:46:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {kubelet Update v1 2021-08-27 20:46:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-08-27 20:48:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-08-27 20:55:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:51:53 +0000 UTC,LastTransitionTime:2021-08-27 20:51:53 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 03:03:44 +0000 UTC,LastTransitionTime:2021-08-27 20:46:51 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 03:03:44 +0000 UTC,LastTransitionTime:2021-08-27 20:46:51 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 03:03:44 +0000 UTC,LastTransitionTime:2021-08-27 20:46:51 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 03:03:44 +0000 UTC,LastTransitionTime:2021-08-27 20:48:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:2835065974a64998811b9acd85de209b,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:309b9155-1e2a-4ebd-900f-bba5abfc3a5d,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[sirot/netperf-latest@sha256:23929b922bb077cb341450fc1c90ae42e6f75da5d7ce61cd1880b08560a5ff85 sirot/netperf-latest:latest],SizeBytes:282025213,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Aug 28 03:03:53.495: INFO:
Logging kubelet events for node master2
Aug 28 03:03:53.497: INFO:
Logging pods the kubelet thinks is on node master2
Aug 28 03:03:53.505: INFO: kube-flannel-4znnq started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded)
Aug 28 03:03:53.505: INFO: Init container install-cni ready: true, restart count 2
Aug 28 03:03:53.505: INFO: Container kube-flannel ready: true, restart count 2
Aug 28 03:03:53.505: INFO: kube-multus-ds-amd64-4mgbk started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.505: INFO: Container kube-multus ready: true, restart count 1
Aug 28 03:03:53.505: INFO: prometheus-operator-5bb8cb9d8f-whr5p started at 2021-08-27 20:59:06 +0000 UTC (0+2 container statuses recorded)
Aug 28 03:03:53.506: INFO: Container kube-rbac-proxy ready: true, restart count 0
Aug 28 03:03:53.506: INFO: Container prometheus-operator ready: true, restart count 0
Aug 28 03:03:53.506: INFO: node-exporter-96jk5 started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded)
Aug 28 03:03:53.506: INFO: Container kube-rbac-proxy ready: true, restart count 0
Aug 28 03:03:53.506: INFO: Container node-exporter ready: true, restart count 0
Aug 28 03:03:53.506: INFO: kube-controller-manager-master2 started at 2021-08-27 20:51:01 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.506: INFO: Container kube-controller-manager ready: true, restart count 2
Aug 28 03:03:53.506: INFO: kube-scheduler-master2 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.506: INFO: Container kube-scheduler ready: true, restart count 2
Aug 28 03:03:53.506: INFO: kube-proxy-b4mn9 started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.506: INFO: Container kube-proxy ready: true, restart count 1
Aug 28 03:03:53.506: INFO: node-feature-discovery-controller-5bf5c49849-zr9zd started at 2021-08-27 20:55:09 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.506: INFO: Container nfd-controller ready: true, restart count 0
Aug 28 03:03:53.506: INFO: kube-apiserver-master2 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.506: INFO: Container kube-apiserver ready: true, restart count 0
W0828 03:03:53.518900 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled.
Aug 28 03:03:53.552: INFO:
Latency metrics for node master2
Aug 28 03:03:53.552: INFO:
Logging node info for node master3
Aug 28 03:03:53.554: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 620b6dba-f2c5-46e9-b2ff-d2f4197167d0 142627 0 2021-08-27 20:47:02 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ce:de:ea:c3:40:4f"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-08-27 20:47:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-08-27 20:47:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-08-27 20:48:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:51:44 +0000 UTC,LastTransitionTime:2021-08-27 20:51:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 03:03:52 +0000 UTC,LastTransitionTime:2021-08-27 20:47:02 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 03:03:52 +0000 UTC,LastTransitionTime:2021-08-27 20:47:02 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 03:03:52 +0000 UTC,LastTransitionTime:2021-08-27 20:47:02 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 03:03:52 +0000 UTC,LastTransitionTime:2021-08-27 20:48:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:37decbffe0e84048b5801289ad3be5bf,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:c96fe8b9-1ce0-44cd-935a-b58987e26570,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496891,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Aug 28 03:03:53.555: INFO:
Logging kubelet events for node master3
Aug 28 03:03:53.557: INFO:
Logging pods the kubelet thinks is on node master3
Aug 28 03:03:53.566: INFO: kube-apiserver-master3 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.566: INFO: Container kube-apiserver ready: true, restart count 0
Aug 28 03:03:53.566: INFO: kube-scheduler-master3 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.566: INFO: Container kube-scheduler ready: true, restart count 2
Aug 28 03:03:53.566: INFO: kube-multus-ds-amd64-wwcgv started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.566: INFO: Container kube-multus ready: true, restart count 1
Aug 28 03:03:53.566: INFO: node-exporter-d4m7q started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded)
Aug 28 03:03:53.566: INFO: Container kube-rbac-proxy ready: true, restart count 0
Aug 28 03:03:53.566: INFO: Container node-exporter ready: true, restart count 0
Aug 28 03:03:53.566: INFO: coredns-7677f9bb54-rxplt started at 2021-08-27 20:49:21 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.566: INFO: Container coredns ready: true, restart count 1
Aug 28 03:03:53.566: INFO: kube-controller-manager-master3 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.566: INFO: Container kube-controller-manager ready: true, restart count 2
Aug 28 03:03:53.566: INFO: kube-proxy-8sxhm started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.566: INFO: Container kube-proxy ready: true, restart count 1
Aug 28 03:03:53.566: INFO: kube-flannel-fkz5d started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded)
Aug 28 03:03:53.566: INFO: Init container install-cni ready: true, restart count 0
Aug 28 03:03:53.566: INFO: Container kube-flannel ready: true, restart count 1
Aug 28 03:03:53.566: INFO: dns-autoscaler-5b7b5c9b6f-54xch started at 2021-08-27 20:49:19 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.566: INFO: Container autoscaler ready: true, restart count 1
W0828 03:03:53.577118 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled.
Aug 28 03:03:53.606: INFO:
Latency metrics for node master3
Aug 28 03:03:53.606: INFO:
Logging node info for node node1
Aug 28 03:03:53.609: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 e7a2481e-32db-4c83-bd9f-4a0687258e7a 142616 0 2021-08-27 20:48:09 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.36.2.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1382":"csi-mock-csi-mock-volumes-1382","csi-mock-csi-mock-volumes-1844":"csi-mock-csi-mock-volumes-1844","csi-mock-csi-mock-volumes-2731":"csi-mock-csi-mock-volumes-2731","csi-mock-csi-mock-volumes-4084":"csi-mock-csi-mock-volumes-4084","csi-mock-csi-mock-volumes-6046":"csi-mock-csi-mock-volumes-6046","csi-mock-csi-mock-volumes-6248":"csi-mock-csi-mock-volumes-6248","csi-mock-csi-mock-volumes-7859":"csi-mock-csi-mock-volumes-7859","csi-mock-csi-mock-volumes-7866":"csi-mock-csi-mock-volumes-7866","csi-mock-csi-mock-volumes-9165":"csi-mock-csi-mock-volumes-9165","csi-mock-csi-mock-volumes-9410":"csi-mock-csi-mock-volumes-9410"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"56:c7:37:40:51:ca"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-08-27 20:48:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-08-27 20:55:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-08-27 20:57:40 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-08-28 02:45:58 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-08-28 03:03:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}},"f:status":{"f:volumesAttached":{}}}} {kubelet Update v1 2021-08-28 03:03:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:io.kubernetes.storage.mock/node":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}},"f:volumesInUse":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {<nil>} 10 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269628928 0} {<nil>} 196552372Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {<nil>} 10 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884628480 0} {<nil>} 174692020Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:51:49 +0000 UTC,LastTransitionTime:2021-08-27 20:51:49 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 03:03:51 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 03:03:51 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 03:03:51 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 03:03:51 +0000 UTC,LastTransitionTime:2021-08-27 20:48:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:c1e38e80ea114a5f96601202301ce842,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:e769e86d-15c0-442c-a93b-bcc6c33ff1cd,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:331c6faa8b0d5ec72cf105e87d35df0a2f2baeec3d6217a51faa73f9460f937f localhost:30500/barometer-collectd:stable],SizeBytes:1238704157,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:a7cea43d9d2f67c38fbf0407786edbe660ee9072945f7bb272b55fd255e8eaca opnfv/barometer-collectd:stable],SizeBytes:1075746799,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:1003787960,},ContainerImage{Names:[localhost:30500/cmk@sha256:fd1487b0c07556a087eff669e70c501a704720dcd53ff75183593de6720585f2 cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[aquasec/kube-hunter@sha256:2be6820bc1d7e0f57193a9a27d5a3e16b2fd93c53747b03ce8ca48c6fc323781 aquasec/kube-hunter:0.3.1],SizeBytes:347611549,},ContainerImage{Names:[golang@sha256:db2475a1dbb2149508e5db31d7d77a75e6600d54be645f37681f03f2762169ba golang:alpine3.12],SizeBytes:301186719,},ContainerImage{Names:[sirot/netperf-latest@sha256:23929b922bb077cb341450fc1c90ae42e6f75da5d7ce61cd1880b08560a5ff85 sirot/netperf-latest:latest],SizeBytes:282025213,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:d7300ccf7ff3e9cea2111d275143b8050618bbc1d1ffe41f46286b1696261243 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44393508,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/regression-issue-74839-amd64@sha256:3b36bd80b97c532a774e7f6246797b8575d97037982f353476c703ba6686c75c gcr.io/kubernetes-e2e-test-images/regression-issue-74839-amd64:1.0],SizeBytes:19227369,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:de25c7fc6c4f3a27c7f0c2dff454e4671823a34d88abd533f210848d527e0fbb alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-7859^4],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-7859^4,DevicePath:,},},Config:nil,},}
Aug 28 03:03:53.610: INFO:
Logging kubelet events for node node1
Aug 28 03:03:53.612: INFO:
Logging pods the kubelet thinks is on node node1
Aug 28 03:03:53.630: INFO: prometheus-k8s-0 started at 2021-08-27 20:59:29 +0000 UTC (0+5 container statuses recorded)
Aug 28 03:03:53.630: INFO: Container custom-metrics-apiserver ready: true, restart count 0
Aug 28 03:03:53.630: INFO: Container grafana ready: true, restart count 0
Aug 28 03:03:53.630: INFO: Container prometheus ready: true, restart count 1
Aug 28 03:03:53.630: INFO: Container prometheus-config-reloader ready: true, restart count 0
Aug 28 03:03:53.630: INFO: Container rules-configmap-reloader ready: true, restart count 0
Aug 28 03:03:53.630: INFO: hostexec-node1-x62tg started at 2021-08-28 03:01:36 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.630: INFO: Container agnhost-container ready: true, restart count 0
Aug 28 03:03:53.630: INFO: hostexec-node1-87vcw started at 2021-08-28 03:03:27 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.630: INFO: Container agnhost-container ready: true, restart count 0
Aug 28 03:03:53.630: INFO: pod-ca18df76-4a2b-4936-b073-952be61a5656 started at 2021-08-28 03:03:37 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.630: INFO: Container write-pod ready: false, restart count 0
Aug 28 03:03:53.630: INFO: hostexec-node1-mt8hz started at 2021-08-28 03:03:42 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.630: INFO: Container agnhost-container ready: true, restart count 0
Aug 28 03:03:53.630: INFO: kube-multus-ds-amd64-nn7bl started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.630: INFO: Container kube-multus ready: true, restart count 1
Aug 28 03:03:53.630: INFO: kubernetes-metrics-scraper-678c97765c-gtp5x started at 2021-08-27 20:49:21 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.630: INFO: Container kubernetes-metrics-scraper ready: true, restart count 1
Aug 28 03:03:53.630: INFO: cmk-init-discover-node1-spg26 started at 2021-08-27 20:57:37 +0000 UTC (0+3 container statuses recorded)
Aug 28 03:03:53.630: INFO: Container discover ready: false, restart count 0
Aug 28 03:03:53.630: INFO: Container init ready: false, restart count 0
Aug 28 03:03:53.630: INFO: Container install ready: false, restart count 0
Aug 28 03:03:53.630: INFO: nginx-proxy-node1 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.630: INFO: Container nginx-proxy ready: true, restart count 2
Aug 28 03:03:53.630: INFO: hostexec-node1-crk62 started at 2021-08-28 03:02:43 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.630: INFO: Container agnhost-container ready: true, restart count 0
Aug 28 03:03:53.630: INFO: csi-mockplugin-attacher-0 started at 2021-08-28 03:03:24 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.630: INFO: Container csi-attacher ready: true, restart count 0
Aug 28 03:03:53.630: INFO: hostexec-node1-5qnj9 started at 2021-08-28 03:03:47 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.630: INFO: Container agnhost-container ready: true, restart count 0
Aug 28 03:03:53.630: INFO: kube-proxy-pb5bl started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.630: INFO: Container kube-proxy ready: true, restart count 2
Aug 28 03:03:53.630: INFO: collectd-ccvwg started at 2021-08-27 21:04:15 +0000 UTC (0+3 container statuses recorded)
Aug 28 03:03:53.630: INFO: Container collectd ready: true, restart count 0
Aug 28 03:03:53.630: INFO: Container collectd-exporter ready: true, restart count 0
Aug 28 03:03:53.630: INFO: Container rbac-proxy ready: true, restart count 0
Aug 28 03:03:53.630: INFO: pod-6897fb2a-0288-434f-a651-175fd71ceffc started at 2021-08-28 03:03:06 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.630: INFO: Container write-pod ready: false, restart count 0
Aug 28 03:03:53.630: INFO: kube-flannel-ssxn7 started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded)
Aug 28 03:03:53.630: INFO: Init container install-cni ready: true, restart count 0
Aug 28 03:03:53.630: INFO: Container kube-flannel ready: true, restart count 1
Aug 28 03:03:53.630: INFO: hostexec-node1-nxs7x started at 2021-08-28 03:02:59 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.630: INFO: Container agnhost-container ready: true, restart count 0
Aug 28 03:03:53.630: INFO: cmk-jw4m6 started at 2021-08-27 20:58:19 +0000 UTC (0+2 container statuses recorded)
Aug 28 03:03:53.630: INFO: Container nodereport ready: true, restart count 0
Aug 28 03:03:53.630: INFO: Container reconcile ready: true, restart count 0
Aug 28 03:03:53.630: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-9lndx started at 2021-08-27 20:55:51 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.630: INFO: Container kube-sriovdp ready: true, restart count 0
Aug 28 03:03:53.630: INFO: node-exporter-4cvlq started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded)
Aug 28 03:03:53.630: INFO: Container kube-rbac-proxy ready: true, restart count 0
Aug 28 03:03:53.630: INFO: Container node-exporter ready: true, restart count 0
Aug 28 03:03:53.630: INFO: pvc-volume-tester-22gsn started at 2021-08-28 03:03:35 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.630: INFO: Container volume-tester ready: false, restart count 0
Aug 28 03:03:53.630: INFO: pod-8b1fd158-bd8f-49f0-8cec-adff8611b81a started at 2021-08-28 03:03:49 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.630: INFO: Container write-pod ready: false, restart count 0
Aug 28 03:03:53.630: INFO: pod-9c953050-6df2-4b46-88fa-b80b115a1dd1 started at 2021-08-28 03:01:52 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.630: INFO: Container write-pod ready: false, restart count 0
Aug 28 03:03:53.630: INFO: pod-c5e38e57-2119-449b-afe1-366b73f753a1 started at 2021-08-28 03:02:53 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.630: INFO: Container write-pod ready: false, restart count 0
Aug 28 03:03:53.630: INFO: pod-8adb3b69-099e-4341-adaf-b41ceffe78d9 started at 2021-08-28 03:03:52 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.630: INFO: Container write-pod ready: false, restart count 0
Aug 28 03:03:53.630: INFO: kubernetes-dashboard-86c6f9df5b-c56fg started at 2021-08-27 20:49:21 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.630: INFO: Container kubernetes-dashboard ready: true, restart count 1
Aug 28 03:03:53.630: INFO: node-feature-discovery-worker-bd9kg started at 2021-08-27 20:55:06 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.630: INFO: Container nfd-worker ready: true, restart count 0
Aug 28 03:03:53.630: INFO: hostexec-node1-ztv9s started at 2021-08-28 02:56:55 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.630: INFO: Container agnhost-container ready: true, restart count 0
Aug 28 03:03:53.630: INFO: csi-mockplugin-0 started at 2021-08-28 03:03:24 +0000 UTC (0+3 container statuses recorded)
Aug 28 03:03:53.630: INFO: Container csi-provisioner ready: true, restart count 0
Aug 28 03:03:53.630: INFO: Container driver-registrar ready: true, restart count 0
Aug 28 03:03:53.630: INFO: Container mock ready: true, restart count 0
W0828 03:03:53.641565 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled.
Aug 28 03:03:53.766: INFO:
Latency metrics for node node1
Aug 28 03:03:53.766: INFO:
Logging node info for node node2
Aug 28 03:03:53.768: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 eb7da7c0-513f-4072-a078-ad3d24f88114 142488 0 2021-08-27 20:48:09 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.36.2.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1442":"csi-mock-csi-mock-volumes-1442","csi-mock-csi-mock-volumes-2831":"csi-mock-csi-mock-volumes-2831","csi-mock-csi-mock-volumes-2967":"csi-mock-csi-mock-volumes-2967","csi-mock-csi-mock-volumes-3285":"csi-mock-csi-mock-volumes-3285","csi-mock-csi-mock-volumes-5585":"csi-mock-csi-mock-volumes-5585","csi-mock-csi-mock-volumes-7451":"csi-mock-csi-mock-volumes-7451","csi-mock-csi-mock-volumes-8521":"csi-mock-csi-mock-volumes-8521"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"9a:88:9c:d1:39:68"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-08-27 20:48:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-08-27 20:55:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-08-27 20:58:03 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-08-28 02:45:47 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-08-28 03:01:49 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubelet Update v1 2021-08-28 03:02:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {<nil>} 10 DecimalSI},example.com/fakecpu: {{1 3} {<nil>} 1k DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269628928 0} {<nil>} 196552372Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {<nil>} 10 DecimalSI},example.com/fakecpu: {{1 3} {<nil>} 1k DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884628480 0} {<nil>} 174692020Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:50:36 +0000 UTC,LastTransitionTime:2021-08-27 20:50:36 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 03:03:45 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 03:03:45 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 03:03:45 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 03:03:45 +0000 UTC,LastTransitionTime:2021-08-27 20:48:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:0ccfc2a4a9b7400c9ca53b5de0ca4970,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:26b02690-0814-4c92-9f6d-d315df796ce6,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:331c6faa8b0d5ec72cf105e87d35df0a2f2baeec3d6217a51faa73f9460f937f localhost:30500/barometer-collectd:stable],SizeBytes:1238704157,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[localhost:30500/cmk@sha256:fd1487b0c07556a087eff669e70c501a704720dcd53ff75183593de6720585f2 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[k8s.gcr.io/sig-storage/nfs-provisioner@sha256:c1bedac8758029948afe060bf8f6ee63ea489b5e08d29745f44fab68ee0d46ca k8s.gcr.io/sig-storage/nfs-provisioner:v2.2.2],SizeBytes:391772778,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:d7300ccf7ff3e9cea2111d275143b8050618bbc1d1ffe41f46286b1696261243 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44393508,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:ae72171f047a37ee5423e0692df7429830919af16e9d668ab0c80b723863d102 localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:cd98d1edca8e5e2e3ea42cbc463812483e5d069d10f0974ca9d484b5a7bd68db localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[aquasec/kube-bench@sha256:3544f6662feb73d36fdba35b17652e2fd73aae45bd4b60e76d7ab928220b3cc6 aquasec/kube-bench:0.3.1],SizeBytes:19301876,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Aug 28 03:03:53.769: INFO:
Logging kubelet events for node node2
Aug 28 03:03:53.771: INFO:
Logging pods the kubelet thinks is on node node2
Aug 28 03:03:53.794: INFO: configmap-client started at 2021-08-28 03:02:53 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.794: INFO: Container configmap-client ready: false, restart count 0
Aug 28 03:03:53.794: INFO: cmk-fzjgr started at 2021-08-27 20:58:20 +0000 UTC (0+2 container statuses recorded)
Aug 28 03:03:53.794: INFO: Container nodereport ready: true, restart count 0
Aug 28 03:03:53.794: INFO: Container reconcile ready: true, restart count 0
Aug 28 03:03:53.794: INFO: local-volume-statefulset-2 started at 2021-08-28 02:57:28 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.794: INFO: Container nginx ready: false, restart count 0
Aug 28 03:03:53.794: INFO: local-volume-statefulset-0 started at 2021-08-28 02:57:28 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.794: INFO: Container nginx ready: false, restart count 0
Aug 28 03:03:53.794: INFO: local-volume-statefulset-1 started at 2021-08-28 02:57:28 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.794: INFO: Container nginx ready: false, restart count 0
Aug 28 03:03:53.794: INFO: collectd-64dp2 started at 2021-08-27 21:04:15 +0000 UTC (0+3 container statuses recorded)
Aug 28 03:03:53.794: INFO: Container collectd ready: true, restart count 0
Aug 28 03:03:53.794: INFO: Container collectd-exporter ready: true, restart count 0
Aug 28 03:03:53.794: INFO: Container rbac-proxy ready: true, restart count 0
Aug 28 03:03:53.794: INFO: cmk-webhook-6c9d5f8578-ndbx2 started at 2021-08-27 20:58:20 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.794: INFO: Container cmk-webhook ready: true, restart count 0
Aug 28 03:03:53.794: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-v99df started at 2021-08-27 21:02:08 +0000 UTC (0+2 container statuses recorded)
Aug 28 03:03:53.794: INFO: Container tas-controller ready: true, restart count 0
Aug 28 03:03:53.794: INFO: Container tas-extender ready: true, restart count 0
Aug 28 03:03:53.794: INFO: kube-flannel-t9qv4 started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded)
Aug 28 03:03:53.794: INFO: Init container install-cni ready: true, restart count 0
Aug 28 03:03:53.794: INFO: Container kube-flannel ready: true, restart count 2
Aug 28 03:03:53.794: INFO: cmk-init-discover-node2-l9qjd started at 2021-08-27 20:57:57 +0000 UTC (0+3 container statuses recorded)
Aug 28 03:03:53.794: INFO: Container discover ready: false, restart count 0
Aug 28 03:03:53.794: INFO: Container init ready: false, restart count 0
Aug 28 03:03:53.794: INFO: Container install ready: false, restart count 0
Aug 28 03:03:53.794: INFO: hostexec-node2-q48ft started at 2021-08-28 02:57:12 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.794: INFO: Container agnhost-container ready: true, restart count 0
Aug 28 03:03:53.794: INFO: node-feature-discovery-worker-54lfh started at 2021-08-27 20:55:06 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.794: INFO: Container nfd-worker ready: true, restart count 0
Aug 28 03:03:53.794: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4f962 started at 2021-08-27 20:55:51 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.794: INFO: Container kube-sriovdp ready: true, restart count 0
Aug 28 03:03:53.794: INFO: kube-multus-ds-amd64-tfffk started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.794: INFO: Container kube-multus ready: true, restart count 1
Aug 28 03:03:53.794: INFO: node-exporter-p6h5h started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded)
Aug 28 03:03:53.794: INFO: Container kube-rbac-proxy ready: true, restart count 0
Aug 28 03:03:53.794: INFO: Container node-exporter ready: true, restart count 0
Aug 28 03:03:53.794: INFO: pod-secrets-92f12714-401d-4089-82b5-6e8014f9ef8e started at 2021-08-28 02:59:07 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.794: INFO: Container creates-volume-test ready: false, restart count 0
Aug 28 03:03:53.794: INFO: nginx-proxy-node2 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.794: INFO: Container nginx-proxy ready: true, restart count 1
Aug 28 03:03:53.794: INFO: kube-proxy-r4q4t started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:03:53.794: INFO: Container kube-proxy ready: true, restart count 2
W0828 03:03:53.805999 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled.
Aug 28 03:03:53.843: INFO:
Latency metrics for node node2
Aug 28 03:03:53.843: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
STEP: Destroying namespace "persistent-local-volumes-test-8743" for this suite.
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:405
Aug 28 03:14:27.927: Failed waiting for pods to enter running: timed out waiting for the condition
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/statefulset/wait.go:58
[BeforeEach] [sig-storage] PersistentVolumes-local
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174
STEP: Creating a kubernetes client
Aug 28 03:04:00.288: INFO: >>> kubeConfig: /root/.kube/config
STEP: Building a namespace api object, basename persistent-local-volumes-test
STEP: Waiting for a default service account to be provisioned in namespace
[BeforeEach] [sig-storage] PersistentVolumes-local
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155
[BeforeEach] StatefulSet with pod affinity [Slow]
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:377
STEP: Setting up local volumes on node "node1"
STEP: Initializing test volumes
Aug 28 03:04:02.336: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-668fd6d6-86b0-42d4-823c-c84f007df778] Namespace:persistent-local-volumes-test-2802 PodName:hostexec-node1-kqdrs ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 03:04:02.336: INFO: >>> kubeConfig: /root/.kube/config
Aug 28 03:04:02.457: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-3a3677d4-b7ca-4af6-b745-78f2319de213] Namespace:persistent-local-volumes-test-2802 PodName:hostexec-node1-kqdrs ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 03:04:02.457: INFO: >>> kubeConfig: /root/.kube/config
Aug 28 03:04:02.576: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-e7121b3b-022f-4db1-8e6a-293c85423c93] Namespace:persistent-local-volumes-test-2802 PodName:hostexec-node1-kqdrs ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 03:04:02.576: INFO: >>> kubeConfig: /root/.kube/config
Aug 28 03:04:02.696: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-62c7b084-4758-47be-bedb-1293124a3f3c] Namespace:persistent-local-volumes-test-2802 PodName:hostexec-node1-kqdrs ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 03:04:02.696: INFO: >>> kubeConfig: /root/.kube/config
Aug 28 03:04:02.809: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-626f3327-af9d-43af-a687-5fda96338da2] Namespace:persistent-local-volumes-test-2802 PodName:hostexec-node1-kqdrs ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 03:04:02.809: INFO: >>> kubeConfig: /root/.kube/config
Aug 28 03:04:02.919: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-4613a2a7-b193-4932-b3f6-a76ca8ce80d7] Namespace:persistent-local-volumes-test-2802 PodName:hostexec-node1-kqdrs ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 03:04:02.919: INFO: >>> kubeConfig: /root/.kube/config
STEP: Creating local PVCs and PVs
Aug 28 03:04:03.035: INFO: Creating a PV followed by a PVC
Aug 28 03:04:03.042: INFO: Creating a PV followed by a PVC
Aug 28 03:04:03.048: INFO: Creating a PV followed by a PVC
Aug 28 03:04:03.055: INFO: Creating a PV followed by a PVC
Aug 28 03:04:03.060: INFO: Creating a PV followed by a PVC
Aug 28 03:04:03.066: INFO: Creating a PV followed by a PVC
Aug 28 03:04:13.112: INFO: PVCs were not bound within 10s (that's good)
STEP: Setting up local volumes on node "node2"
STEP: Initializing test volumes
Aug 28 03:04:17.128: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-0b8ed9bb-6dbe-4711-b407-ae923313ba8b] Namespace:persistent-local-volumes-test-2802 PodName:hostexec-node2-4sgh9 ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 03:04:17.128: INFO: >>> kubeConfig: /root/.kube/config
Aug 28 03:04:17.253: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-320a4ae6-d6fe-4e13-8d81-268570d8d0b2] Namespace:persistent-local-volumes-test-2802 PodName:hostexec-node2-4sgh9 ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 03:04:17.253: INFO: >>> kubeConfig: /root/.kube/config
Aug 28 03:04:17.371: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-777449ea-f3e1-4cb4-b7e4-b6f053eb6d80] Namespace:persistent-local-volumes-test-2802 PodName:hostexec-node2-4sgh9 ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 03:04:17.371: INFO: >>> kubeConfig: /root/.kube/config
Aug 28 03:04:17.488: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-929291c3-57f9-44b8-8dc6-73fcd3a4ee5a] Namespace:persistent-local-volumes-test-2802 PodName:hostexec-node2-4sgh9 ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 03:04:17.488: INFO: >>> kubeConfig: /root/.kube/config
Aug 28 03:04:17.606: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-eb617fa4-9f1e-4b1c-b66f-e20e376631ae] Namespace:persistent-local-volumes-test-2802 PodName:hostexec-node2-4sgh9 ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 03:04:17.606: INFO: >>> kubeConfig: /root/.kube/config
Aug 28 03:04:17.723: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-90582180-1f3a-4d35-bf5d-fcc4062bfb70] Namespace:persistent-local-volumes-test-2802 PodName:hostexec-node2-4sgh9 ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 03:04:17.723: INFO: >>> kubeConfig: /root/.kube/config
STEP: Creating local PVCs and PVs
Aug 28 03:04:17.840: INFO: Creating a PV followed by a PVC
Aug 28 03:04:17.847: INFO: Creating a PV followed by a PVC
Aug 28 03:04:17.853: INFO: Creating a PV followed by a PVC
Aug 28 03:04:17.858: INFO: Creating a PV followed by a PVC
Aug 28 03:04:17.864: INFO: Creating a PV followed by a PVC
Aug 28 03:04:17.869: INFO: Creating a PV followed by a PVC
Aug 28 03:04:27.911: INFO: PVCs were not bound within 10s (that's good)
[It] should use volumes on one node when pod has affinity
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:405
STEP: Creating a StatefulSet with pod affinity on nodes
Aug 28 03:04:27.918: INFO: Found 0 stateful pods, waiting for 3
Aug 28 03:04:37.922: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:04:47.922: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:04:57.922: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:05:07.923: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:05:17.922: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:05:27.923: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:05:37.922: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:05:47.923: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:05:57.923: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:06:07.923: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:06:17.922: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:06:27.923: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:06:37.925: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:06:47.923: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:06:57.922: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:07:07.922: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:07:17.922: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:07:27.925: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:07:37.921: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:07:47.922: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:07:57.922: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:08:07.923: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:08:17.923: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:08:27.922: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:08:37.923: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:08:47.924: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:08:57.922: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:09:07.922: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:09:17.923: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:09:27.922: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:09:37.922: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:09:47.923: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:09:57.923: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:10:07.923: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:10:17.923: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:10:27.922: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:10:37.922: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:10:47.923: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:10:57.923: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:11:07.922: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:11:17.923: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:11:27.922: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:11:37.922: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:11:47.921: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:11:57.924: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:12:07.924: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:12:17.923: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:12:27.923: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:12:37.923: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:12:47.922: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:12:57.922: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:13:07.922: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:13:17.923: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:13:27.922: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:13:37.922: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:13:47.923: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:13:57.924: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:14:07.925: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:14:17.924: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:14:27.924: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:14:27.927: INFO: Found 1 stateful pods, waiting for 3
Aug 28 03:14:27.927: FAIL: Failed waiting for pods to enter running: timed out waiting for the condition
Full Stack Trace
k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning(0x5411460, 0xc001b442c0, 0x300000003, 0xc00051d400)
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/statefulset/wait.go:58 +0x10e
k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/statefulset/wait.go:80
k8s.io/kubernetes/test/e2e/storage.createStatefulSet(0xc002129880, 0x3, 0x2, 0x0, 0x0)
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:1198 +0xbf1
k8s.io/kubernetes/test/e2e/storage.glob..func20.5.4()
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:407 +0x7d
k8s.io/kubernetes/test/e2e.RunE2ETests(0xc0038dcc00)
_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345
k8s.io/kubernetes/test/e2e.TestE2E(0xc0038dcc00)
_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b
testing.tRunner(0xc0038dcc00, 0x4dec428)
/usr/local/go/src/testing/testing.go:1123 +0xef
created by testing.(*T).Run
/usr/local/go/src/testing/testing.go:1168 +0x2b3
[AfterEach] StatefulSet with pod affinity [Slow]
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:389
STEP: Cleaning up PVC and PV
Aug 28 03:14:27.929: INFO: Deleting PersistentVolumeClaim "pvc-dkpwf"
Aug 28 03:14:27.933: INFO: Deleting PersistentVolume "local-pvgz69x"
STEP: Cleaning up PVC and PV
Aug 28 03:14:27.937: INFO: Deleting PersistentVolumeClaim "pvc-v4gwf"
Aug 28 03:14:27.941: INFO: Deleting PersistentVolume "local-pv6fz67"
STEP: Cleaning up PVC and PV
Aug 28 03:14:27.946: INFO: Deleting PersistentVolumeClaim "pvc-jq4md"
Aug 28 03:14:27.953: INFO: Deleting PersistentVolume "local-pvjj29j"
STEP: Cleaning up PVC and PV
Aug 28 03:14:27.959: INFO: Deleting PersistentVolumeClaim "pvc-9zqk5"
Aug 28 03:14:27.965: INFO: Deleting PersistentVolume "local-pv9dssk"
STEP: Cleaning up PVC and PV
Aug 28 03:14:27.968: INFO: Deleting PersistentVolumeClaim "pvc-f2kf2"
Aug 28 03:14:27.972: INFO: Deleting PersistentVolume "local-pvt5tws"
STEP: Cleaning up PVC and PV
Aug 28 03:14:27.975: INFO: Deleting PersistentVolumeClaim "pvc-nvhsl"
Aug 28 03:14:27.978: INFO: Deleting PersistentVolume "local-pvkn642"
STEP: Removing the test directory
Aug 28 03:14:27.982: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-668fd6d6-86b0-42d4-823c-c84f007df778] Namespace:persistent-local-volumes-test-2802 PodName:hostexec-node1-kqdrs ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 03:14:27.982: INFO: >>> kubeConfig: /root/.kube/config
STEP: Removing the test directory
Aug 28 03:14:28.105: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-3a3677d4-b7ca-4af6-b745-78f2319de213] Namespace:persistent-local-volumes-test-2802 PodName:hostexec-node1-kqdrs ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 03:14:28.105: INFO: >>> kubeConfig: /root/.kube/config
STEP: Removing the test directory
Aug 28 03:14:28.225: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-e7121b3b-022f-4db1-8e6a-293c85423c93] Namespace:persistent-local-volumes-test-2802 PodName:hostexec-node1-kqdrs ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 03:14:28.225: INFO: >>> kubeConfig: /root/.kube/config
STEP: Removing the test directory
Aug 28 03:14:28.343: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-62c7b084-4758-47be-bedb-1293124a3f3c] Namespace:persistent-local-volumes-test-2802 PodName:hostexec-node1-kqdrs ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 03:14:28.343: INFO: >>> kubeConfig: /root/.kube/config
STEP: Removing the test directory
Aug 28 03:14:28.458: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-626f3327-af9d-43af-a687-5fda96338da2] Namespace:persistent-local-volumes-test-2802 PodName:hostexec-node1-kqdrs ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 03:14:28.458: INFO: >>> kubeConfig: /root/.kube/config
STEP: Removing the test directory
Aug 28 03:14:28.582: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-4613a2a7-b193-4932-b3f6-a76ca8ce80d7] Namespace:persistent-local-volumes-test-2802 PodName:hostexec-node1-kqdrs ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 03:14:28.582: INFO: >>> kubeConfig: /root/.kube/config
STEP: Cleaning up PVC and PV
Aug 28 03:14:28.700: INFO: Deleting PersistentVolumeClaim "pvc-xbd5t"
Aug 28 03:14:28.705: INFO: Deleting PersistentVolume "local-pvk5s7v"
STEP: Cleaning up PVC and PV
Aug 28 03:14:28.708: INFO: Deleting PersistentVolumeClaim "pvc-p9rvf"
Aug 28 03:14:28.712: INFO: Deleting PersistentVolume "local-pvdgwj9"
STEP: Cleaning up PVC and PV
Aug 28 03:14:28.716: INFO: Deleting PersistentVolumeClaim "pvc-j688t"
Aug 28 03:14:28.718: INFO: Deleting PersistentVolume "local-pv4nhmq"
STEP: Cleaning up PVC and PV
Aug 28 03:14:28.722: INFO: Deleting PersistentVolumeClaim "pvc-glbg8"
Aug 28 03:14:28.725: INFO: Deleting PersistentVolume "local-pvfb9wm"
STEP: Cleaning up PVC and PV
Aug 28 03:14:28.728: INFO: Deleting PersistentVolumeClaim "pvc-s429f"
Aug 28 03:14:28.732: INFO: Deleting PersistentVolume "local-pv4t446"
STEP: Cleaning up PVC and PV
Aug 28 03:14:28.736: INFO: Deleting PersistentVolumeClaim "pvc-skn45"
Aug 28 03:14:28.739: INFO: Deleting PersistentVolume "local-pvkj4kd"
STEP: Removing the test directory
Aug 28 03:14:28.743: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-0b8ed9bb-6dbe-4711-b407-ae923313ba8b] Namespace:persistent-local-volumes-test-2802 PodName:hostexec-node2-4sgh9 ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 03:14:28.743: INFO: >>> kubeConfig: /root/.kube/config
STEP: Removing the test directory
Aug 28 03:14:28.868: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-320a4ae6-d6fe-4e13-8d81-268570d8d0b2] Namespace:persistent-local-volumes-test-2802 PodName:hostexec-node2-4sgh9 ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 03:14:28.868: INFO: >>> kubeConfig: /root/.kube/config
STEP: Removing the test directory
Aug 28 03:14:29.000: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-777449ea-f3e1-4cb4-b7e4-b6f053eb6d80] Namespace:persistent-local-volumes-test-2802 PodName:hostexec-node2-4sgh9 ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 03:14:29.000: INFO: >>> kubeConfig: /root/.kube/config
STEP: Removing the test directory
Aug 28 03:14:29.115: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-929291c3-57f9-44b8-8dc6-73fcd3a4ee5a] Namespace:persistent-local-volumes-test-2802 PodName:hostexec-node2-4sgh9 ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 03:14:29.115: INFO: >>> kubeConfig: /root/.kube/config
STEP: Removing the test directory
Aug 28 03:14:29.228: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-eb617fa4-9f1e-4b1c-b66f-e20e376631ae] Namespace:persistent-local-volumes-test-2802 PodName:hostexec-node2-4sgh9 ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 03:14:29.228: INFO: >>> kubeConfig: /root/.kube/config
STEP: Removing the test directory
Aug 28 03:14:29.339: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-90582180-1f3a-4d35-bf5d-fcc4062bfb70] Namespace:persistent-local-volumes-test-2802 PodName:hostexec-node2-4sgh9 ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}
Aug 28 03:14:29.339: INFO: >>> kubeConfig: /root/.kube/config
[AfterEach] [sig-storage] PersistentVolumes-local
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175
STEP: Collecting events from namespace "persistent-local-volumes-test-2802".
STEP: Found 34 events.
Aug 28 03:14:29.460: INFO: At 2021-08-28 03:04:00 +0000 UTC - event for hostexec-node1-kqdrs: {default-scheduler } Scheduled: Successfully assigned persistent-local-volumes-test-2802/hostexec-node1-kqdrs to node1
Aug 28 03:14:29.460: INFO: At 2021-08-28 03:04:00 +0000 UTC - event for hostexec-node1-kqdrs: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20"
Aug 28 03:14:29.460: INFO: At 2021-08-28 03:04:01 +0000 UTC - event for hostexec-node1-kqdrs: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 443.460185ms
Aug 28 03:14:29.460: INFO: At 2021-08-28 03:04:01 +0000 UTC - event for hostexec-node1-kqdrs: {kubelet node1} Created: Created container agnhost-container
Aug 28 03:14:29.460: INFO: At 2021-08-28 03:04:01 +0000 UTC - event for hostexec-node1-kqdrs: {kubelet node1} Started: Started container agnhost-container
Aug 28 03:14:29.460: INFO: At 2021-08-28 03:04:03 +0000 UTC - event for pvc-9zqk5: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Aug 28 03:14:29.460: INFO: At 2021-08-28 03:04:03 +0000 UTC - event for pvc-dkpwf: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Aug 28 03:14:29.460: INFO: At 2021-08-28 03:04:03 +0000 UTC - event for pvc-f2kf2: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Aug 28 03:14:29.460: INFO: At 2021-08-28 03:04:03 +0000 UTC - event for pvc-jq4md: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Aug 28 03:14:29.460: INFO: At 2021-08-28 03:04:03 +0000 UTC - event for pvc-nvhsl: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Aug 28 03:14:29.460: INFO: At 2021-08-28 03:04:03 +0000 UTC - event for pvc-v4gwf: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Aug 28 03:14:29.460: INFO: At 2021-08-28 03:04:13 +0000 UTC - event for hostexec-node2-4sgh9: {kubelet node2} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20"
Aug 28 03:14:29.460: INFO: At 2021-08-28 03:04:13 +0000 UTC - event for hostexec-node2-4sgh9: {default-scheduler } Scheduled: Successfully assigned persistent-local-volumes-test-2802/hostexec-node2-4sgh9 to node2
Aug 28 03:14:29.460: INFO: At 2021-08-28 03:04:14 +0000 UTC - event for hostexec-node2-4sgh9: {kubelet node2} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 554.710973ms
Aug 28 03:14:29.460: INFO: At 2021-08-28 03:04:14 +0000 UTC - event for hostexec-node2-4sgh9: {kubelet node2} Created: Created container agnhost-container
Aug 28 03:14:29.460: INFO: At 2021-08-28 03:04:15 +0000 UTC - event for hostexec-node2-4sgh9: {kubelet node2} Started: Started container agnhost-container
Aug 28 03:14:29.460: INFO: At 2021-08-28 03:04:17 +0000 UTC - event for pvc-glbg8: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Aug 28 03:14:29.460: INFO: At 2021-08-28 03:04:17 +0000 UTC - event for pvc-j688t: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Aug 28 03:14:29.460: INFO: At 2021-08-28 03:04:17 +0000 UTC - event for pvc-p9rvf: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Aug 28 03:14:29.460: INFO: At 2021-08-28 03:04:17 +0000 UTC - event for pvc-s429f: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Aug 28 03:14:29.460: INFO: At 2021-08-28 03:04:17 +0000 UTC - event for pvc-skn45: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Aug 28 03:14:29.460: INFO: At 2021-08-28 03:04:17 +0000 UTC - event for pvc-xbd5t: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Aug 28 03:14:29.460: INFO: At 2021-08-28 03:04:27 +0000 UTC - event for local-volume-statefulset: {statefulset-controller } SuccessfulCreate: create Claim vol2-local-volume-statefulset-0 Pod local-volume-statefulset-0 in StatefulSet local-volume-statefulset success
Aug 28 03:14:29.460: INFO: At 2021-08-28 03:04:27 +0000 UTC - event for local-volume-statefulset: {statefulset-controller } SuccessfulCreate: create Pod local-volume-statefulset-0 in StatefulSet local-volume-statefulset successful
Aug 28 03:14:29.460: INFO: At 2021-08-28 03:04:27 +0000 UTC - event for local-volume-statefulset: {statefulset-controller } SuccessfulCreate: create Claim vol1-local-volume-statefulset-0 Pod local-volume-statefulset-0 in StatefulSet local-volume-statefulset success
Aug 28 03:14:29.460: INFO: At 2021-08-28 03:04:27 +0000 UTC - event for vol1-local-volume-statefulset-0: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Aug 28 03:14:29.460: INFO: At 2021-08-28 03:04:27 +0000 UTC - event for vol2-local-volume-statefulset-0: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Aug 28 03:14:29.460: INFO: At 2021-08-28 03:04:28 +0000 UTC - event for local-volume-statefulset-0: {default-scheduler } Scheduled: Successfully assigned persistent-local-volumes-test-2802/local-volume-statefulset-0 to node2
Aug 28 03:14:29.460: INFO: At 2021-08-28 03:04:31 +0000 UTC - event for local-volume-statefulset-0: {kubelet node2} Pulling: Pulling image "docker.io/library/nginx:1.14-alpine"
Aug 28 03:14:29.460: INFO: At 2021-08-28 03:04:31 +0000 UTC - event for local-volume-statefulset-0: {multus } AddedInterface: Add eth0 [10.244.4.222/24]
Aug 28 03:14:29.460: INFO: At 2021-08-28 03:04:32 +0000 UTC - event for local-volume-statefulset-0: {kubelet node2} Failed: Failed to pull image "docker.io/library/nginx:1.14-alpine": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit
Aug 28 03:14:29.460: INFO: At 2021-08-28 03:04:32 +0000 UTC - event for local-volume-statefulset-0: {kubelet node2} Failed: Error: ErrImagePull
Aug 28 03:14:29.460: INFO: At 2021-08-28 03:04:33 +0000 UTC - event for local-volume-statefulset-0: {kubelet node2} Failed: Error: ImagePullBackOff
Aug 28 03:14:29.460: INFO: At 2021-08-28 03:04:33 +0000 UTC - event for local-volume-statefulset-0: {kubelet node2} BackOff: Back-off pulling image "docker.io/library/nginx:1.14-alpine"
Aug 28 03:14:29.463: INFO: POD NODE PHASE GRACE CONDITIONS
Aug 28 03:14:29.463: INFO: hostexec-node1-kqdrs node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 03:04:00 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 03:04:01 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 03:04:01 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 03:04:00 +0000 UTC }]
Aug 28 03:14:29.463: INFO: hostexec-node2-4sgh9 node2 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 03:04:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 03:04:16 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 03:04:16 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 03:04:13 +0000 UTC }]
Aug 28 03:14:29.463: INFO: local-volume-statefulset-0 node2 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 03:04:28 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-08-28 03:04:28 +0000 UTC ContainersNotReady containers with unready status: [nginx]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-08-28 03:04:28 +0000 UTC ContainersNotReady containers with unready status: [nginx]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-08-28 03:04:28 +0000 UTC }]
Aug 28 03:14:29.463: INFO:
Aug 28 03:14:29.467: INFO:
Logging node info for node master1
Aug 28 03:14:29.469: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 3af53387-5aee-42c1-b0e7-644cf9161d48 149699 0 2021-08-27 20:46:13 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"92:f8:b6:72:e4:be"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-08-27 20:46:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-08-27 20:46:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {kube-controller-manager Update v1 2021-08-27 20:46:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:50:56 +0000 UTC,LastTransitionTime:2021-08-27 20:50:56 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 03:14:28 +0000 UTC,LastTransitionTime:2021-08-27 20:46:13 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 03:14:28 +0000 UTC,LastTransitionTime:2021-08-27 20:46:13 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 03:14:28 +0000 UTC,LastTransitionTime:2021-08-27 20:46:13 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 03:14:28 +0000 UTC,LastTransitionTime:2021-08-27 20:50:30 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:5d584023135a46ecb77596bf48ed7f2f,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:587cafa0-6de3-49f8-906e-06315a8ff104,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[sirot/netperf-latest@sha256:23929b922bb077cb341450fc1c90ae42e6f75da5d7ce61cd1880b08560a5ff85 sirot/netperf-latest:latest],SizeBytes:282025213,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:ae72171f047a37ee5423e0692df7429830919af16e9d668ab0c80b723863d102 tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:cd98d1edca8e5e2e3ea42cbc463812483e5d069d10f0974ca9d484b5a7bd68db tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[aquasec/kube-bench@sha256:3544f6662feb73d36fdba35b17652e2fd73aae45bd4b60e76d7ab928220b3cc6 aquasec/kube-bench:0.3.1],SizeBytes:19301876,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Aug 28 03:14:29.470: INFO:
Logging kubelet events for node master1
Aug 28 03:14:29.472: INFO:
Logging pods the kubelet thinks is on node master1
Aug 28 03:14:29.485: INFO: kube-scheduler-master1 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.485: INFO: Container kube-scheduler ready: true, restart count 0
Aug 28 03:14:29.485: INFO: kube-flannel-pp7vp started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded)
Aug 28 03:14:29.485: INFO: Init container install-cni ready: true, restart count 0
Aug 28 03:14:29.485: INFO: Container kube-flannel ready: true, restart count 3
Aug 28 03:14:29.485: INFO: kube-multus-ds-amd64-sfr9k started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.485: INFO: Container kube-multus ready: true, restart count 1
Aug 28 03:14:29.485: INFO: docker-registry-docker-registry-56cbc7bc58-cthtt started at 2021-08-27 20:51:47 +0000 UTC (0+2 container statuses recorded)
Aug 28 03:14:29.485: INFO: Container docker-registry ready: true, restart count 0
Aug 28 03:14:29.485: INFO: Container nginx ready: true, restart count 0
Aug 28 03:14:29.485: INFO: kube-apiserver-master1 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.485: INFO: Container kube-apiserver ready: true, restart count 0
Aug 28 03:14:29.485: INFO: kube-controller-manager-master1 started at 2021-08-27 20:54:27 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.485: INFO: Container kube-controller-manager ready: true, restart count 3
Aug 28 03:14:29.485: INFO: kube-proxy-rb5p6 started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.485: INFO: Container kube-proxy ready: true, restart count 1
Aug 28 03:14:29.485: INFO: coredns-7677f9bb54-dwtp5 started at 2021-08-27 20:49:16 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.485: INFO: Container coredns ready: true, restart count 1
Aug 28 03:14:29.485: INFO: node-exporter-z2ngr started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded)
Aug 28 03:14:29.485: INFO: Container kube-rbac-proxy ready: true, restart count 0
Aug 28 03:14:29.485: INFO: Container node-exporter ready: true, restart count 0
W0828 03:14:29.499247 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled.
Aug 28 03:14:29.525: INFO:
Latency metrics for node master1
Aug 28 03:14:29.525: INFO:
Logging node info for node master2
Aug 28 03:14:29.528: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 666e473f-d9e6-4c06-8b56-06474d788f70 149672 0 2021-08-27 20:46:51 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"42:86:ff:30:bd:4d"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-08-27 20:46:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {kubelet Update v1 2021-08-27 20:46:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-08-27 20:48:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-08-27 20:55:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:51:53 +0000 UTC,LastTransitionTime:2021-08-27 20:51:53 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 03:14:26 +0000 UTC,LastTransitionTime:2021-08-27 20:46:51 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 03:14:26 +0000 UTC,LastTransitionTime:2021-08-27 20:46:51 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 03:14:26 +0000 UTC,LastTransitionTime:2021-08-27 20:46:51 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 03:14:26 +0000 UTC,LastTransitionTime:2021-08-27 20:48:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:2835065974a64998811b9acd85de209b,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:309b9155-1e2a-4ebd-900f-bba5abfc3a5d,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[sirot/netperf-latest@sha256:23929b922bb077cb341450fc1c90ae42e6f75da5d7ce61cd1880b08560a5ff85 sirot/netperf-latest:latest],SizeBytes:282025213,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Aug 28 03:14:29.529: INFO:
Logging kubelet events for node master2
Aug 28 03:14:29.531: INFO:
Logging pods the kubelet thinks is on node master2
Aug 28 03:14:29.543: INFO: kube-scheduler-master2 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.543: INFO: Container kube-scheduler ready: true, restart count 2
Aug 28 03:14:29.543: INFO: kube-proxy-b4mn9 started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.543: INFO: Container kube-proxy ready: true, restart count 1
Aug 28 03:14:29.543: INFO: node-feature-discovery-controller-5bf5c49849-zr9zd started at 2021-08-27 20:55:09 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.543: INFO: Container nfd-controller ready: true, restart count 0
Aug 28 03:14:29.543: INFO: kube-apiserver-master2 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.543: INFO: Container kube-apiserver ready: true, restart count 0
Aug 28 03:14:29.543: INFO: kube-flannel-4znnq started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded)
Aug 28 03:14:29.543: INFO: Init container install-cni ready: true, restart count 2
Aug 28 03:14:29.543: INFO: Container kube-flannel ready: true, restart count 2
Aug 28 03:14:29.543: INFO: kube-multus-ds-amd64-4mgbk started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.543: INFO: Container kube-multus ready: true, restart count 1
Aug 28 03:14:29.543: INFO: prometheus-operator-5bb8cb9d8f-whr5p started at 2021-08-27 20:59:06 +0000 UTC (0+2 container statuses recorded)
Aug 28 03:14:29.543: INFO: Container kube-rbac-proxy ready: true, restart count 0
Aug 28 03:14:29.543: INFO: Container prometheus-operator ready: true, restart count 0
Aug 28 03:14:29.543: INFO: node-exporter-96jk5 started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded)
Aug 28 03:14:29.543: INFO: Container kube-rbac-proxy ready: true, restart count 0
Aug 28 03:14:29.543: INFO: Container node-exporter ready: true, restart count 0
Aug 28 03:14:29.543: INFO: kube-controller-manager-master2 started at 2021-08-27 20:51:01 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.543: INFO: Container kube-controller-manager ready: true, restart count 2
W0828 03:14:29.557044 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled.
Aug 28 03:14:29.583: INFO:
Latency metrics for node master2
Aug 28 03:14:29.583: INFO:
Logging node info for node master3
Aug 28 03:14:29.586: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 620b6dba-f2c5-46e9-b2ff-d2f4197167d0 149662 0 2021-08-27 20:47:02 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ce:de:ea:c3:40:4f"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-08-27 20:47:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-08-27 20:47:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-08-27 20:48:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:51:44 +0000 UTC,LastTransitionTime:2021-08-27 20:51:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 03:14:24 +0000 UTC,LastTransitionTime:2021-08-27 20:47:02 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 03:14:24 +0000 UTC,LastTransitionTime:2021-08-27 20:47:02 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 03:14:24 +0000 UTC,LastTransitionTime:2021-08-27 20:47:02 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 03:14:24 +0000 UTC,LastTransitionTime:2021-08-27 20:48:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:37decbffe0e84048b5801289ad3be5bf,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:c96fe8b9-1ce0-44cd-935a-b58987e26570,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496891,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Aug 28 03:14:29.586: INFO:
Logging kubelet events for node master3
Aug 28 03:14:29.588: INFO:
Logging pods the kubelet thinks is on node master3
Aug 28 03:14:29.602: INFO: kube-apiserver-master3 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.603: INFO: Container kube-apiserver ready: true, restart count 0
Aug 28 03:14:29.603: INFO: kube-scheduler-master3 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.603: INFO: Container kube-scheduler ready: true, restart count 2
Aug 28 03:14:29.603: INFO: kube-multus-ds-amd64-wwcgv started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.603: INFO: Container kube-multus ready: true, restart count 1
Aug 28 03:14:29.603: INFO: node-exporter-d4m7q started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded)
Aug 28 03:14:29.603: INFO: Container kube-rbac-proxy ready: true, restart count 0
Aug 28 03:14:29.603: INFO: Container node-exporter ready: true, restart count 0
Aug 28 03:14:29.603: INFO: coredns-7677f9bb54-rxplt started at 2021-08-27 20:49:21 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.603: INFO: Container coredns ready: true, restart count 1
Aug 28 03:14:29.603: INFO: kube-controller-manager-master3 started at 2021-08-27 20:47:27 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.603: INFO: Container kube-controller-manager ready: true, restart count 2
Aug 28 03:14:29.603: INFO: kube-proxy-8sxhm started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.603: INFO: Container kube-proxy ready: true, restart count 1
Aug 28 03:14:29.603: INFO: kube-flannel-fkz5d started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded)
Aug 28 03:14:29.603: INFO: Init container install-cni ready: true, restart count 0
Aug 28 03:14:29.603: INFO: Container kube-flannel ready: true, restart count 1
Aug 28 03:14:29.603: INFO: dns-autoscaler-5b7b5c9b6f-54xch started at 2021-08-27 20:49:19 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.603: INFO: Container autoscaler ready: true, restart count 1
W0828 03:14:29.614268 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled.
Aug 28 03:14:29.645: INFO:
Latency metrics for node master3
Aug 28 03:14:29.645: INFO:
Logging node info for node node1
Aug 28 03:14:29.648: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 e7a2481e-32db-4c83-bd9f-4a0687258e7a 149664 0 2021-08-27 20:48:09 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.36.2.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1382":"csi-mock-csi-mock-volumes-1382","csi-mock-csi-mock-volumes-1844":"csi-mock-csi-mock-volumes-1844","csi-mock-csi-mock-volumes-2691":"csi-mock-csi-mock-volumes-2691","csi-mock-csi-mock-volumes-2731":"csi-mock-csi-mock-volumes-2731","csi-mock-csi-mock-volumes-4084":"csi-mock-csi-mock-volumes-4084","csi-mock-csi-mock-volumes-6046":"csi-mock-csi-mock-volumes-6046","csi-mock-csi-mock-volumes-6248":"csi-mock-csi-mock-volumes-6248","csi-mock-csi-mock-volumes-6493":"csi-mock-csi-mock-volumes-6493","csi-mock-csi-mock-volumes-7027":"csi-mock-csi-mock-volumes-7027","csi-mock-csi-mock-volumes-7859":"csi-mock-csi-mock-volumes-7859","csi-mock-csi-mock-volumes-7866":"csi-mock-csi-mock-volumes-7866","csi-mock-csi-mock-volumes-9157":"csi-mock-csi-mock-volumes-9157","csi-mock-csi-mock-volumes-9165":"csi-mock-csi-mock-volumes-9165","csi-mock-csi-mock-volumes-9410":"csi-mock-csi-mock-volumes-9410"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"56:c7:37:40:51:ca"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-08-27 20:48:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-08-27 20:55:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-08-27 20:57:40 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-08-28 02:45:58 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-08-28 03:10:43 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubelet Update v1 2021-08-28 03:10:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:io.kubernetes.storage.mock/node":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {<nil>} 10 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269628928 0} {<nil>} 196552372Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {<nil>} 10 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884628480 0} {<nil>} 174692020Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:51:49 +0000 UTC,LastTransitionTime:2021-08-27 20:51:49 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 03:14:25 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 03:14:25 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 03:14:25 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 03:14:25 +0000 UTC,LastTransitionTime:2021-08-27 20:48:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:c1e38e80ea114a5f96601202301ce842,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:e769e86d-15c0-442c-a93b-bcc6c33ff1cd,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:331c6faa8b0d5ec72cf105e87d35df0a2f2baeec3d6217a51faa73f9460f937f localhost:30500/barometer-collectd:stable],SizeBytes:1238704157,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:a7cea43d9d2f67c38fbf0407786edbe660ee9072945f7bb272b55fd255e8eaca opnfv/barometer-collectd:stable],SizeBytes:1075746799,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:1003787960,},ContainerImage{Names:[localhost:30500/cmk@sha256:fd1487b0c07556a087eff669e70c501a704720dcd53ff75183593de6720585f2 cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[aquasec/kube-hunter@sha256:2be6820bc1d7e0f57193a9a27d5a3e16b2fd93c53747b03ce8ca48c6fc323781 aquasec/kube-hunter:0.3.1],SizeBytes:347611549,},ContainerImage{Names:[golang@sha256:db2475a1dbb2149508e5db31d7d77a75e6600d54be645f37681f03f2762169ba golang:alpine3.12],SizeBytes:301186719,},ContainerImage{Names:[sirot/netperf-latest@sha256:23929b922bb077cb341450fc1c90ae42e6f75da5d7ce61cd1880b08560a5ff85 sirot/netperf-latest:latest],SizeBytes:282025213,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:d7300ccf7ff3e9cea2111d275143b8050618bbc1d1ffe41f46286b1696261243 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44393508,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/regression-issue-74839-amd64@sha256:3b36bd80b97c532a774e7f6246797b8575d97037982f353476c703ba6686c75c gcr.io/kubernetes-e2e-test-images/regression-issue-74839-amd64:1.0],SizeBytes:19227369,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:de25c7fc6c4f3a27c7f0c2dff454e4671823a34d88abd533f210848d527e0fbb alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Aug 28 03:14:29.649: INFO:
Logging kubelet events for node node1
Aug 28 03:14:29.651: INFO:
Logging pods the kubelet thinks is on node node1
Aug 28 03:14:29.672: INFO: cmk-jw4m6 started at 2021-08-27 20:58:19 +0000 UTC (0+2 container statuses recorded)
Aug 28 03:14:29.672: INFO: Container nodereport ready: true, restart count 0
Aug 28 03:14:29.672: INFO: Container reconcile ready: true, restart count 0
Aug 28 03:14:29.672: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-9lndx started at 2021-08-27 20:55:51 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.672: INFO: Container kube-sriovdp ready: true, restart count 0
Aug 28 03:14:29.672: INFO: node-exporter-4cvlq started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded)
Aug 28 03:14:29.672: INFO: Container kube-rbac-proxy ready: true, restart count 0
Aug 28 03:14:29.672: INFO: Container node-exporter ready: true, restart count 0
Aug 28 03:14:29.672: INFO: kubernetes-dashboard-86c6f9df5b-c56fg started at 2021-08-27 20:49:21 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.672: INFO: Container kubernetes-dashboard ready: true, restart count 1
Aug 28 03:14:29.672: INFO: node-feature-discovery-worker-bd9kg started at 2021-08-27 20:55:06 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.672: INFO: Container nfd-worker ready: true, restart count 0
Aug 28 03:14:29.672: INFO: prometheus-k8s-0 started at 2021-08-27 20:59:29 +0000 UTC (0+5 container statuses recorded)
Aug 28 03:14:29.673: INFO: Container custom-metrics-apiserver ready: true, restart count 0
Aug 28 03:14:29.673: INFO: Container grafana ready: true, restart count 0
Aug 28 03:14:29.673: INFO: Container prometheus ready: true, restart count 1
Aug 28 03:14:29.673: INFO: Container prometheus-config-reloader ready: true, restart count 0
Aug 28 03:14:29.673: INFO: Container rules-configmap-reloader ready: true, restart count 0
Aug 28 03:14:29.673: INFO: kube-multus-ds-amd64-nn7bl started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.673: INFO: Container kube-multus ready: true, restart count 1
Aug 28 03:14:29.673: INFO: kubernetes-metrics-scraper-678c97765c-gtp5x started at 2021-08-27 20:49:21 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.673: INFO: Container kubernetes-metrics-scraper ready: true, restart count 1
Aug 28 03:14:29.673: INFO: cmk-init-discover-node1-spg26 started at 2021-08-27 20:57:37 +0000 UTC (0+3 container statuses recorded)
Aug 28 03:14:29.673: INFO: Container discover ready: false, restart count 0
Aug 28 03:14:29.673: INFO: Container init ready: false, restart count 0
Aug 28 03:14:29.673: INFO: Container install ready: false, restart count 0
Aug 28 03:14:29.673: INFO: nginx-proxy-node1 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.673: INFO: Container nginx-proxy ready: true, restart count 2
Aug 28 03:14:29.673: INFO: hostexec-node1-kqdrs started at 2021-08-28 03:04:00 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.673: INFO: Container agnhost-container ready: true, restart count 0
Aug 28 03:14:29.673: INFO: kube-proxy-pb5bl started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.673: INFO: Container kube-proxy ready: true, restart count 2
Aug 28 03:14:29.673: INFO: collectd-ccvwg started at 2021-08-27 21:04:15 +0000 UTC (0+3 container statuses recorded)
Aug 28 03:14:29.673: INFO: Container collectd ready: true, restart count 0
Aug 28 03:14:29.673: INFO: Container collectd-exporter ready: true, restart count 0
Aug 28 03:14:29.673: INFO: Container rbac-proxy ready: true, restart count 0
Aug 28 03:14:29.673: INFO: kube-flannel-ssxn7 started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded)
Aug 28 03:14:29.673: INFO: Init container install-cni ready: true, restart count 0
Aug 28 03:14:29.673: INFO: Container kube-flannel ready: true, restart count 1
W0828 03:14:29.685655 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled.
Aug 28 03:14:29.759: INFO:
Latency metrics for node node1
Aug 28 03:14:29.759: INFO:
Logging node info for node node2
Aug 28 03:14:29.762: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 eb7da7c0-513f-4072-a078-ad3d24f88114 149704 0 2021-08-27 20:48:09 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.36.2.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1442":"csi-mock-csi-mock-volumes-1442","csi-mock-csi-mock-volumes-1936":"csi-mock-csi-mock-volumes-1936","csi-mock-csi-mock-volumes-2652":"csi-mock-csi-mock-volumes-2652","csi-mock-csi-mock-volumes-2831":"csi-mock-csi-mock-volumes-2831","csi-mock-csi-mock-volumes-2967":"csi-mock-csi-mock-volumes-2967","csi-mock-csi-mock-volumes-3285":"csi-mock-csi-mock-volumes-3285","csi-mock-csi-mock-volumes-5585":"csi-mock-csi-mock-volumes-5585","csi-mock-csi-mock-volumes-7451":"csi-mock-csi-mock-volumes-7451","csi-mock-csi-mock-volumes-8521":"csi-mock-csi-mock-volumes-8521"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"9a:88:9c:d1:39:68"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-08-27 20:48:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-08-27 20:48:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-08-27 20:55:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-08-27 20:58:03 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-08-28 02:45:47 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-08-28 03:01:49 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubelet Update v1 2021-08-28 03:02:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {<nil>} 10 DecimalSI},example.com/fakecpu: {{1 3} {<nil>} 1k DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269628928 0} {<nil>} 196552372Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {<nil>} 10 DecimalSI},example.com/fakecpu: {{1 3} {<nil>} 1k DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884628480 0} {<nil>} 174692020Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-08-27 20:50:36 +0000 UTC,LastTransitionTime:2021-08-27 20:50:36 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-08-28 03:14:28 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-08-28 03:14:28 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-08-28 03:14:28 +0000 UTC,LastTransitionTime:2021-08-27 20:48:09 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-08-28 03:14:28 +0000 UTC,LastTransitionTime:2021-08-27 20:48:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:0ccfc2a4a9b7400c9ca53b5de0ca4970,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:26b02690-0814-4c92-9f6d-d315df796ce6,KernelVersion:3.10.0-1160.36.2.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:331c6faa8b0d5ec72cf105e87d35df0a2f2baeec3d6217a51faa73f9460f937f localhost:30500/barometer-collectd:stable],SizeBytes:1238704157,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[localhost:30500/cmk@sha256:fd1487b0c07556a087eff669e70c501a704720dcd53ff75183593de6720585f2 localhost:30500/cmk:v1.5.1],SizeBytes:723496952,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[k8s.gcr.io/sig-storage/nfs-provisioner@sha256:c1bedac8758029948afe060bf8f6ee63ea489b5e08d29745f44fab68ee0d46ca k8s.gcr.io/sig-storage/nfs-provisioner:v2.2.2],SizeBytes:391772778,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:d7300ccf7ff3e9cea2111d275143b8050618bbc1d1ffe41f46286b1696261243 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44393508,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:ae72171f047a37ee5423e0692df7429830919af16e9d668ab0c80b723863d102 localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:cd98d1edca8e5e2e3ea42cbc463812483e5d069d10f0974ca9d484b5a7bd68db localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[aquasec/kube-bench@sha256:3544f6662feb73d36fdba35b17652e2fd73aae45bd4b60e76d7ab928220b3cc6 aquasec/kube-bench:0.3.1],SizeBytes:19301876,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Aug 28 03:14:29.763: INFO:
Logging kubelet events for node node2
Aug 28 03:14:29.765: INFO:
Logging pods the kubelet thinks is on node node2
Aug 28 03:14:29.786: INFO: cmk-fzjgr started at 2021-08-27 20:58:20 +0000 UTC (0+2 container statuses recorded)
Aug 28 03:14:29.786: INFO: Container nodereport ready: true, restart count 0
Aug 28 03:14:29.786: INFO: Container reconcile ready: true, restart count 0
Aug 28 03:14:29.786: INFO: collectd-64dp2 started at 2021-08-27 21:04:15 +0000 UTC (0+3 container statuses recorded)
Aug 28 03:14:29.786: INFO: Container collectd ready: true, restart count 0
Aug 28 03:14:29.786: INFO: Container collectd-exporter ready: true, restart count 0
Aug 28 03:14:29.786: INFO: Container rbac-proxy ready: true, restart count 0
Aug 28 03:14:29.786: INFO: local-volume-statefulset-0 started at 2021-08-28 03:04:28 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.786: INFO: Container nginx ready: false, restart count 0
Aug 28 03:14:29.786: INFO: cmk-webhook-6c9d5f8578-ndbx2 started at 2021-08-27 20:58:20 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.786: INFO: Container cmk-webhook ready: true, restart count 0
Aug 28 03:14:29.786: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-v99df started at 2021-08-27 21:02:08 +0000 UTC (0+2 container statuses recorded)
Aug 28 03:14:29.786: INFO: Container tas-controller ready: true, restart count 0
Aug 28 03:14:29.786: INFO: Container tas-extender ready: true, restart count 0
Aug 28 03:14:29.786: INFO: kube-flannel-t9qv4 started at 2021-08-27 20:48:48 +0000 UTC (1+1 container statuses recorded)
Aug 28 03:14:29.786: INFO: Init container install-cni ready: true, restart count 0
Aug 28 03:14:29.786: INFO: Container kube-flannel ready: true, restart count 2
Aug 28 03:14:29.786: INFO: cmk-init-discover-node2-l9qjd started at 2021-08-27 20:57:57 +0000 UTC (0+3 container statuses recorded)
Aug 28 03:14:29.786: INFO: Container discover ready: false, restart count 0
Aug 28 03:14:29.786: INFO: Container init ready: false, restart count 0
Aug 28 03:14:29.786: INFO: Container install ready: false, restart count 0
Aug 28 03:14:29.786: INFO: node-feature-discovery-worker-54lfh started at 2021-08-27 20:55:06 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.786: INFO: Container nfd-worker ready: true, restart count 0
Aug 28 03:14:29.786: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4f962 started at 2021-08-27 20:55:51 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.786: INFO: Container kube-sriovdp ready: true, restart count 0
Aug 28 03:14:29.786: INFO: hostexec-node2-4sgh9 started at 2021-08-28 03:04:13 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.786: INFO: Container agnhost-container ready: true, restart count 0
Aug 28 03:14:29.786: INFO: nginx-proxy-node2 started at 2021-08-27 20:54:17 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.786: INFO: Container nginx-proxy ready: true, restart count 1
Aug 28 03:14:29.786: INFO: kube-proxy-r4q4t started at 2021-08-27 20:48:12 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.786: INFO: Container kube-proxy ready: true, restart count 2
Aug 28 03:14:29.786: INFO: kube-multus-ds-amd64-tfffk started at 2021-08-27 20:48:56 +0000 UTC (0+1 container statuses recorded)
Aug 28 03:14:29.786: INFO: Container kube-multus ready: true, restart count 1
Aug 28 03:14:29.786: INFO: node-exporter-p6h5h started at 2021-08-27 20:59:13 +0000 UTC (0+2 container statuses recorded)
Aug 28 03:14:29.786: INFO: Container kube-rbac-proxy ready: true, restart count 0
Aug 28 03:14:29.786: INFO: Container node-exporter ready: true, restart count 0
W0828 03:14:29.800544 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled.
Aug 28 03:14:29.840: INFO:
Latency metrics for node node2
Aug 28 03:14:29.840: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
STEP: Destroying namespace "persistent-local-volumes-test-2802" for this suite.