Running Suite: Kubernetes e2e suite =================================== Random Seed: 1621043036 - Will randomize all specs Will run 5484 specs Running in parallel across 10 nodes May 15 01:43:58.164: INFO: >>> kubeConfig: /root/.kube/config May 15 01:43:58.169: INFO: Waiting up to 30m0s for all (but 0) nodes to be schedulable May 15 01:43:58.193: INFO: Waiting up to 10m0s for all pods (need at least 0) in namespace 'kube-system' to be running and ready May 15 01:43:58.259: INFO: The status of Pod cmk-init-discover-node2-j75ff is Succeeded, skipping waiting May 15 01:43:58.259: INFO: 40 / 41 pods in namespace 'kube-system' are running and ready (0 seconds elapsed) May 15 01:43:58.259: INFO: expected 8 pod replicas in namespace 'kube-system', 8 are Running and Ready. May 15 01:43:58.259: INFO: Waiting up to 5m0s for all daemonsets in namespace 'kube-system' to start May 15 01:43:58.276: INFO: 2 / 2 pods ready in namespace 'kube-system' in daemonset 'cmk' (0 seconds elapsed) May 15 01:43:58.276: INFO: 5 / 5 pods ready in namespace 'kube-system' in daemonset 'kube-flannel' (0 seconds elapsed) May 15 01:43:58.276: INFO: 0 / 0 pods ready in namespace 'kube-system' in daemonset 'kube-flannel-ds-arm' (0 seconds elapsed) May 15 01:43:58.276: INFO: 0 / 0 pods ready in namespace 'kube-system' in daemonset 'kube-flannel-ds-arm64' (0 seconds elapsed) May 15 01:43:58.276: INFO: 0 / 0 pods ready in namespace 'kube-system' in daemonset 'kube-flannel-ds-ppc64le' (0 seconds elapsed) May 15 01:43:58.276: INFO: 0 / 0 pods ready in namespace 'kube-system' in daemonset 'kube-flannel-ds-s390x' (0 seconds elapsed) May 15 01:43:58.276: INFO: 5 / 5 pods ready in namespace 'kube-system' in daemonset 'kube-multus-ds-amd64' (0 seconds elapsed) May 15 01:43:58.276: INFO: 5 / 5 pods ready in namespace 'kube-system' in daemonset 'kube-proxy' (0 seconds elapsed) May 15 01:43:58.276: INFO: 2 / 2 pods ready in namespace 'kube-system' in daemonset 'node-feature-discovery-worker' (0 seconds elapsed) May 15 01:43:58.276: INFO: 2 / 2 pods ready in namespace 'kube-system' in daemonset 'sriov-net-dp-kube-sriov-device-plugin-amd64' (0 seconds elapsed) May 15 01:43:58.276: INFO: e2e test version: v1.19.10 May 15 01:43:58.277: INFO: kube-apiserver version: v1.19.8 May 15 01:43:58.278: INFO: >>> kubeConfig: /root/.kube/config May 15 01:43:58.284: INFO: Cluster IP family: ipv4 SSSSSSS ------------------------------ May 15 01:43:58.280: INFO: >>> kubeConfig: /root/.kube/config May 15 01:43:58.301: INFO: Cluster IP family: ipv4 SSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ May 15 01:43:58.305: INFO: >>> kubeConfig: /root/.kube/config May 15 01:43:58.328: INFO: Cluster IP family: ipv4 SSSS ------------------------------ May 15 01:43:58.309: INFO: >>> kubeConfig: /root/.kube/config May 15 01:43:58.331: INFO: Cluster IP family: ipv4 S ------------------------------ May 15 01:43:58.310: INFO: >>> kubeConfig: /root/.kube/config May 15 01:43:58.332: INFO: Cluster IP family: ipv4 SSSSSSS ------------------------------ May 15 01:43:58.315: INFO: >>> kubeConfig: /root/.kube/config May 15 01:43:58.337: INFO: Cluster IP family: ipv4 SSSSSSSS ------------------------------ May 15 01:43:58.320: INFO: >>> kubeConfig: /root/.kube/config May 15 01:43:58.343: INFO: Cluster IP family: ipv4 May 15 01:43:58.319: INFO: >>> kubeConfig: /root/.kube/config May 15 01:43:58.342: INFO: Cluster IP family: ipv4 SSSSSSSSS ------------------------------ May 15 01:43:58.325: INFO: >>> kubeConfig: /root/.kube/config May 15 01:43:58.346: INFO: Cluster IP family: ipv4 SSSSSSSSS ------------------------------ May 15 01:43:58.325: INFO: >>> kubeConfig: /root/.kube/config May 15 01:43:58.350: INFO: Cluster IP family: ipv4 SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-scheduling] Multi-AZ Cluster Volumes [sig-storage] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:43:58.380: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename multi-az May 15 01:43:58.411: INFO: Found PodSecurityPolicies; testing pod creation to see if PodSecurityPolicy is enabled May 15 01:43:58.413: INFO: Error creating dryrun pod; assuming PodSecurityPolicy is disabled: admission webhook "cmk.intel.com" does not support dry run STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-scheduling] Multi-AZ Cluster Volumes [sig-storage] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduling/ubernetes_lite_volumes.go:44 May 15 01:43:58.415: INFO: Only supported for providers [gce gke] (not skeleton) [AfterEach] [sig-scheduling] Multi-AZ Cluster Volumes [sig-storage] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:43:58.416: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "multi-az-3053" for this suite. S [SKIPPING] in Spec Setup (BeforeEach) [0.045 seconds] [sig-scheduling] Multi-AZ Cluster Volumes [sig-storage] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduling/framework.go:40 should only be allowed to provision PDs in zones where nodes exist [BeforeEach] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduling/ubernetes_lite_volumes.go:59 Only supported for providers [gce gke] (not skeleton) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduling/ubernetes_lite_volumes.go:45 ------------------------------ SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PVC Protection /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:43:58.430: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename pvc-protection May 15 01:43:58.447: INFO: Found PodSecurityPolicies; testing pod creation to see if PodSecurityPolicy is enabled May 15 01:43:58.449: INFO: Error creating dryrun pod; assuming PodSecurityPolicy is disabled: admission webhook "cmk.intel.com" does not support dry run STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PVC Protection /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/pvc_protection.go:71 May 15 01:43:58.451: INFO: Waiting up to 30m0s for all (but 0) nodes to be schedulable STEP: Creating a PVC May 15 01:43:58.456: INFO: error finding default storageClass : No default storage class found [AfterEach] [sig-storage] PVC Protection /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:43:58.457: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "pvc-protection-3999" for this suite. [AfterEach] [sig-storage] PVC Protection /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/pvc_protection.go:106 S [SKIPPING] in Spec Setup (BeforeEach) [0.036 seconds] [sig-storage] PVC Protection /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 Verify that PVC in active use by a pod is not removed immediately [BeforeEach] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/pvc_protection.go:124 error finding default storageClass : No default storage class found /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/pv/pv.go:825 ------------------------------ SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PVC Protection /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:43:58.511: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename pvc-protection May 15 01:43:59.262: INFO: Found PodSecurityPolicies; testing pod creation to see if PodSecurityPolicy is enabled May 15 01:43:59.265: INFO: Error creating dryrun pod; assuming PodSecurityPolicy is disabled: admission webhook "cmk.intel.com" does not support dry run STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PVC Protection /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/pvc_protection.go:71 May 15 01:43:59.267: INFO: Waiting up to 30m0s for all (but 0) nodes to be schedulable STEP: Creating a PVC May 15 01:43:59.272: INFO: error finding default storageClass : No default storage class found [AfterEach] [sig-storage] PVC Protection /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:43:59.274: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "pvc-protection-3236" for this suite. [AfterEach] [sig-storage] PVC Protection /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/pvc_protection.go:106 S [SKIPPING] in Spec Setup (BeforeEach) [0.770 seconds] [sig-storage] PVC Protection /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 Verify that scheduling of a pod that uses PVC that is being deleted fails and the pod becomes Unschedulable [BeforeEach] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/pvc_protection.go:143 error finding default storageClass : No default storage class found /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/pv/pv.go:825 ------------------------------ SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes GCEPD /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:43:59.718: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename pv STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes GCEPD /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-gce.go:77 May 15 01:43:59.739: INFO: Only supported for providers [gce gke] (not skeleton) [AfterEach] [sig-storage] PersistentVolumes GCEPD /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:43:59.740: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "pv-5958" for this suite. [AfterEach] [sig-storage] PersistentVolumes GCEPD /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-gce.go:110 May 15 01:43:59.749: INFO: AfterEach: Cleaning up test resources May 15 01:43:59.749: INFO: pvc is nil May 15 01:43:59.749: INFO: pv is nil S [SKIPPING] in Spec Setup (BeforeEach) [0.031 seconds] [sig-storage] PersistentVolumes GCEPD /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 should test that deleting the Namespace of a PVC and Pod causes the successful detach of Persistent Disk [BeforeEach] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-gce.go:155 Only supported for providers [gce gke] (not skeleton) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-gce.go:85 ------------------------------ SSSSSSSSS ------------------------------ [BeforeEach] [sig-api-machinery] ResourceQuota /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:43:58.386: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename resourcequota May 15 01:43:58.416: INFO: Found PodSecurityPolicies; testing pod creation to see if PodSecurityPolicy is enabled May 15 01:43:58.418: INFO: Error creating dryrun pod; assuming PodSecurityPolicy is disabled: admission webhook "cmk.intel.com" does not support dry run STEP: Waiting for a default service account to be provisioned in namespace [It] should create a ResourceQuota and capture the life of a persistent volume claim with a storage class. [sig-storage] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/apimachinery/resource_quota.go:516 STEP: Counting existing ResourceQuota STEP: Creating a ResourceQuota STEP: Ensuring resource quota status is calculated STEP: Creating a PersistentVolumeClaim with storage class STEP: Ensuring resource quota status captures persistent volume claim creation STEP: Deleting a PersistentVolumeClaim STEP: Ensuring resource quota status released usage [AfterEach] [sig-api-machinery] ResourceQuota /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:44:09.448: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "resourcequota-49" for this suite. • [SLOW TEST:11.071 seconds] [sig-api-machinery] ResourceQuota /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/apimachinery/framework.go:23 should create a ResourceQuota and capture the life of a persistent volume claim with a storage class. [sig-storage] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/apimachinery/resource_quota.go:516 ------------------------------ {"msg":"PASSED [sig-api-machinery] ResourceQuota should create a ResourceQuota and capture the life of a persistent volume claim with a storage class. [sig-storage]","total":-1,"completed":1,"skipped":15,"failed":0} SSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-api-machinery] ResourceQuota /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:43:58.455: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename resourcequota May 15 01:43:58.563: INFO: Found PodSecurityPolicies; testing pod creation to see if PodSecurityPolicy is enabled May 15 01:43:58.565: INFO: Error creating dryrun pod; assuming PodSecurityPolicy is disabled: admission webhook "cmk.intel.com" does not support dry run STEP: Waiting for a default service account to be provisioned in namespace [It] should create a ResourceQuota and capture the life of a persistent volume claim. [sig-storage] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/apimachinery/resource_quota.go:466 STEP: Counting existing ResourceQuota STEP: Creating a ResourceQuota STEP: Ensuring resource quota status is calculated STEP: Creating a PersistentVolumeClaim STEP: Ensuring resource quota status captures persistent volume claim creation STEP: Deleting a PersistentVolumeClaim STEP: Ensuring resource quota status released usage [AfterEach] [sig-api-machinery] ResourceQuota /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:44:09.597: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "resourcequota-8899" for this suite. • [SLOW TEST:11.151 seconds] [sig-api-machinery] ResourceQuota /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/apimachinery/framework.go:23 should create a ResourceQuota and capture the life of a persistent volume claim. [sig-storage] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/apimachinery/resource_quota.go:466 ------------------------------ {"msg":"PASSED [sig-api-machinery] ResourceQuota should create a ResourceQuota and capture the life of a persistent volume claim. [sig-storage]","total":-1,"completed":1,"skipped":43,"failed":0} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] Projected configMap /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:43:58.527: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename projected May 15 01:43:59.411: INFO: Found PodSecurityPolicies; testing pod creation to see if PodSecurityPolicy is enabled May 15 01:43:59.413: INFO: Error creating dryrun pod; assuming PodSecurityPolicy is disabled: admission webhook "cmk.intel.com" does not support dry run STEP: Waiting for a default service account to be provisioned in namespace [It] should be consumable from pods in volume as non-root with defaultMode and fsGroup set [LinuxOnly] [NodeFeature:FSGroup] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/projected_configmap.go:59 STEP: Creating configMap with name projected-configmap-test-volume-07503bfb-6315-47f6-8933-e0ac4deeb603 STEP: Creating a pod to test consume configMaps May 15 01:43:59.431: INFO: Waiting up to 5m0s for pod "pod-projected-configmaps-632526c1-9827-4ffb-9298-8455825443c4" in namespace "projected-981" to be "Succeeded or Failed" May 15 01:43:59.434: INFO: Pod "pod-projected-configmaps-632526c1-9827-4ffb-9298-8455825443c4": Phase="Pending", Reason="", readiness=false. Elapsed: 3.001269ms May 15 01:44:01.437: INFO: Pod "pod-projected-configmaps-632526c1-9827-4ffb-9298-8455825443c4": Phase="Pending", Reason="", readiness=false. Elapsed: 2.005963486s May 15 01:44:03.440: INFO: Pod "pod-projected-configmaps-632526c1-9827-4ffb-9298-8455825443c4": Phase="Pending", Reason="", readiness=false. Elapsed: 4.008938569s May 15 01:44:05.443: INFO: Pod "pod-projected-configmaps-632526c1-9827-4ffb-9298-8455825443c4": Phase="Pending", Reason="", readiness=false. Elapsed: 6.011522817s May 15 01:44:07.445: INFO: Pod "pod-projected-configmaps-632526c1-9827-4ffb-9298-8455825443c4": Phase="Pending", Reason="", readiness=false. Elapsed: 8.014330213s May 15 01:44:09.448: INFO: Pod "pod-projected-configmaps-632526c1-9827-4ffb-9298-8455825443c4": Phase="Pending", Reason="", readiness=false. Elapsed: 10.017086143s May 15 01:44:11.452: INFO: Pod "pod-projected-configmaps-632526c1-9827-4ffb-9298-8455825443c4": Phase="Succeeded", Reason="", readiness=false. Elapsed: 12.021064592s STEP: Saw pod success May 15 01:44:11.452: INFO: Pod "pod-projected-configmaps-632526c1-9827-4ffb-9298-8455825443c4" satisfied condition "Succeeded or Failed" May 15 01:44:11.454: INFO: Trying to get logs from node node2 pod pod-projected-configmaps-632526c1-9827-4ffb-9298-8455825443c4 container projected-configmap-volume-test: STEP: delete the pod May 15 01:44:11.474: INFO: Waiting for pod pod-projected-configmaps-632526c1-9827-4ffb-9298-8455825443c4 to disappear May 15 01:44:11.476: INFO: Pod pod-projected-configmaps-632526c1-9827-4ffb-9298-8455825443c4 no longer exists [AfterEach] [sig-storage] Projected configMap /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:44:11.476: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "projected-981" for this suite. • [SLOW TEST:12.957 seconds] [sig-storage] Projected configMap /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/projected_configmap.go:36 should be consumable from pods in volume as non-root with defaultMode and fsGroup set [LinuxOnly] [NodeFeature:FSGroup] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/projected_configmap.go:59 ------------------------------ {"msg":"PASSED [sig-storage] Projected configMap should be consumable from pods in volume as non-root with defaultMode and fsGroup set [LinuxOnly] [NodeFeature:FSGroup]","total":-1,"completed":1,"skipped":73,"failed":0} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] Flexvolumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:44:11.589: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename flexvolume STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] Flexvolumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/flexvolume.go:169 May 15 01:44:11.608: INFO: Only supported for providers [gce local] (not skeleton) [AfterEach] [sig-storage] Flexvolumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:44:11.610: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "flexvolume-5253" for this suite. S [SKIPPING] in Spec Setup (BeforeEach) [0.029 seconds] [sig-storage] Flexvolumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 should be mountable when non-attachable [BeforeEach] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/flexvolume.go:188 Only supported for providers [gce local] (not skeleton) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/flexvolume.go:170 ------------------------------ SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] Projected downwardAPI /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:43:59.771: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename projected STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] Projected downwardAPI /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/projected_downwardapi.go:42 [It] should provide podname as non-root with fsgroup and defaultMode [LinuxOnly] [NodeFeature:FSGroup] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/projected_downwardapi.go:107 STEP: Creating a pod to test downward API volume plugin May 15 01:43:59.804: INFO: Waiting up to 5m0s for pod "metadata-volume-b0a696ff-bde6-40c4-9fa9-5b36252c1598" in namespace "projected-8835" to be "Succeeded or Failed" May 15 01:43:59.807: INFO: Pod "metadata-volume-b0a696ff-bde6-40c4-9fa9-5b36252c1598": Phase="Pending", Reason="", readiness=false. Elapsed: 2.416631ms May 15 01:44:01.810: INFO: Pod "metadata-volume-b0a696ff-bde6-40c4-9fa9-5b36252c1598": Phase="Pending", Reason="", readiness=false. Elapsed: 2.006117836s May 15 01:44:03.814: INFO: Pod "metadata-volume-b0a696ff-bde6-40c4-9fa9-5b36252c1598": Phase="Pending", Reason="", readiness=false. Elapsed: 4.009860763s May 15 01:44:05.819: INFO: Pod "metadata-volume-b0a696ff-bde6-40c4-9fa9-5b36252c1598": Phase="Pending", Reason="", readiness=false. Elapsed: 6.014340794s May 15 01:44:07.825: INFO: Pod "metadata-volume-b0a696ff-bde6-40c4-9fa9-5b36252c1598": Phase="Pending", Reason="", readiness=false. Elapsed: 8.020482615s May 15 01:44:09.827: INFO: Pod "metadata-volume-b0a696ff-bde6-40c4-9fa9-5b36252c1598": Phase="Pending", Reason="", readiness=false. Elapsed: 10.023001957s May 15 01:44:11.830: INFO: Pod "metadata-volume-b0a696ff-bde6-40c4-9fa9-5b36252c1598": Phase="Succeeded", Reason="", readiness=false. Elapsed: 12.02612234s STEP: Saw pod success May 15 01:44:11.830: INFO: Pod "metadata-volume-b0a696ff-bde6-40c4-9fa9-5b36252c1598" satisfied condition "Succeeded or Failed" May 15 01:44:11.832: INFO: Trying to get logs from node node2 pod metadata-volume-b0a696ff-bde6-40c4-9fa9-5b36252c1598 container client-container: STEP: delete the pod May 15 01:44:12.455: INFO: Waiting for pod metadata-volume-b0a696ff-bde6-40c4-9fa9-5b36252c1598 to disappear May 15 01:44:12.457: INFO: Pod metadata-volume-b0a696ff-bde6-40c4-9fa9-5b36252c1598 no longer exists [AfterEach] [sig-storage] Projected downwardAPI /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:44:12.457: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "projected-8835" for this suite. • [SLOW TEST:12.695 seconds] [sig-storage] Projected downwardAPI /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/projected_downwardapi.go:36 should provide podname as non-root with fsgroup and defaultMode [LinuxOnly] [NodeFeature:FSGroup] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/projected_downwardapi.go:107 ------------------------------ {"msg":"PASSED [sig-storage] Projected downwardAPI should provide podname as non-root with fsgroup and defaultMode [LinuxOnly] [NodeFeature:FSGroup]","total":-1,"completed":1,"skipped":305,"failed":0} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] Secrets /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:44:12.758: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename secrets STEP: Waiting for a default service account to be provisioned in namespace [It] should be immutable if `immutable` field is set /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/secrets_volume.go:376 [AfterEach] [sig-storage] Secrets /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:44:12.818: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "secrets-4626" for this suite. • ------------------------------ {"msg":"PASSED [sig-storage] Secrets should be immutable if `immutable` field is set","total":-1,"completed":2,"skipped":463,"failed":0} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:43:58.437: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: tmpfs] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes STEP: Creating tmpfs mount point on node "node1" at path "/tmp/local-volume-test-0be34194-61c6-4b8b-975b-506fa8806fee" May 15 01:44:04.481: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p "/tmp/local-volume-test-0be34194-61c6-4b8b-975b-506fa8806fee" && mount -t tmpfs -o size=10m tmpfs-"/tmp/local-volume-test-0be34194-61c6-4b8b-975b-506fa8806fee" "/tmp/local-volume-test-0be34194-61c6-4b8b-975b-506fa8806fee"] Namespace:persistent-local-volumes-test-6101 PodName:hostexec-node1-m75cm ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:44:04.481: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:44:04.610: INFO: Creating a PV followed by a PVC May 15 01:44:04.617: INFO: Waiting for PV local-pvqkj4w to bind to PVC pvc-wsqbp May 15 01:44:04.617: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-wsqbp] to have phase Bound May 15 01:44:04.620: INFO: PersistentVolumeClaim pvc-wsqbp found but phase is Pending instead of Bound. May 15 01:44:06.623: INFO: PersistentVolumeClaim pvc-wsqbp found and phase=Bound (2.005425452s) May 15 01:44:06.623: INFO: Waiting up to 3m0s for PersistentVolume local-pvqkj4w to have phase Bound May 15 01:44:06.625: INFO: PersistentVolume local-pvqkj4w found and phase=Bound (2.5761ms) [BeforeEach] One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:211 STEP: Creating pod1 STEP: Creating a pod May 15 01:44:12.653: INFO: pod "pod-424d6c89-08fe-475f-b77d-fcbe2b82caff" created on Node "node1" STEP: Writing in pod1 May 15 01:44:12.653: INFO: ExecWithOptions {Command:[/bin/sh -c mkdir -p /mnt/volume1; echo test-file-content > /mnt/volume1/test-file] Namespace:persistent-local-volumes-test-6101 PodName:pod-424d6c89-08fe-475f-b77d-fcbe2b82caff ContainerName:write-pod Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:false} May 15 01:44:12.653: INFO: >>> kubeConfig: /root/.kube/config May 15 01:44:12.776: INFO: podRWCmdExec cmd: "mkdir -p /mnt/volume1; echo test-file-content > /mnt/volume1/test-file", out: "", stderr: "", err: [It] should be able to mount volume and write from pod1 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:234 May 15 01:44:12.776: INFO: ExecWithOptions {Command:[/bin/sh -c cat /mnt/volume1/test-file] Namespace:persistent-local-volumes-test-6101 PodName:pod-424d6c89-08fe-475f-b77d-fcbe2b82caff ContainerName:write-pod Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:false} May 15 01:44:12.776: INFO: >>> kubeConfig: /root/.kube/config May 15 01:44:12.881: INFO: podRWCmdExec cmd: "cat /mnt/volume1/test-file", out: "test-file-content", stderr: "", err: STEP: Writing in pod1 May 15 01:44:12.881: INFO: ExecWithOptions {Command:[/bin/sh -c mkdir -p /mnt/volume1; echo /tmp/local-volume-test-0be34194-61c6-4b8b-975b-506fa8806fee > /mnt/volume1/test-file] Namespace:persistent-local-volumes-test-6101 PodName:pod-424d6c89-08fe-475f-b77d-fcbe2b82caff ContainerName:write-pod Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:false} May 15 01:44:12.881: INFO: >>> kubeConfig: /root/.kube/config May 15 01:44:12.985: INFO: podRWCmdExec cmd: "mkdir -p /mnt/volume1; echo /tmp/local-volume-test-0be34194-61c6-4b8b-975b-506fa8806fee > /mnt/volume1/test-file", out: "", stderr: "", err: [AfterEach] One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:223 STEP: Deleting pod1 STEP: Deleting pod pod-424d6c89-08fe-475f-b77d-fcbe2b82caff in namespace persistent-local-volumes-test-6101 [AfterEach] [Volume type: tmpfs] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:44:12.990: INFO: Deleting PersistentVolumeClaim "pvc-wsqbp" May 15 01:44:12.993: INFO: Deleting PersistentVolume "local-pvqkj4w" STEP: Unmount tmpfs mount point on node "node1" at path "/tmp/local-volume-test-0be34194-61c6-4b8b-975b-506fa8806fee" May 15 01:44:12.997: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c umount "/tmp/local-volume-test-0be34194-61c6-4b8b-975b-506fa8806fee"] Namespace:persistent-local-volumes-test-6101 PodName:hostexec-node1-m75cm ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:44:12.997: INFO: >>> kubeConfig: /root/.kube/config STEP: Removing the test directory May 15 01:44:13.112: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-0be34194-61c6-4b8b-975b-506fa8806fee] Namespace:persistent-local-volumes-test-6101 PodName:hostexec-node1-m75cm ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:44:13.112: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:44:13.252: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-6101" for this suite. • [SLOW TEST:14.824 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: tmpfs] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:205 should be able to mount volume and write from pod1 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:234 ------------------------------ {"msg":"PASSED [sig-storage] PersistentVolumes-local [Volume type: tmpfs] One pod requesting one prebound PVC should be able to mount volume and write from pod1","total":-1,"completed":1,"skipped":41,"failed":0} S ------------------------------ [BeforeEach] [sig-storage] EmptyDir volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:44:10.093: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename emptydir STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] when FSGroup is specified [LinuxOnly] [NodeFeature:FSGroup] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/empty_dir.go:47 [It] files with FSGroup ownership should support (root,0644,tmpfs) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/empty_dir.go:64 STEP: Creating a pod to test emptydir 0644 on tmpfs May 15 01:44:10.123: INFO: Waiting up to 5m0s for pod "pod-5198098d-eef6-4689-a6d7-1b38b55f2186" in namespace "emptydir-848" to be "Succeeded or Failed" May 15 01:44:10.127: INFO: Pod "pod-5198098d-eef6-4689-a6d7-1b38b55f2186": Phase="Pending", Reason="", readiness=false. Elapsed: 3.645808ms May 15 01:44:12.131: INFO: Pod "pod-5198098d-eef6-4689-a6d7-1b38b55f2186": Phase="Pending", Reason="", readiness=false. Elapsed: 2.00751914s May 15 01:44:14.136: INFO: Pod "pod-5198098d-eef6-4689-a6d7-1b38b55f2186": Phase="Succeeded", Reason="", readiness=false. Elapsed: 4.012693921s STEP: Saw pod success May 15 01:44:14.136: INFO: Pod "pod-5198098d-eef6-4689-a6d7-1b38b55f2186" satisfied condition "Succeeded or Failed" May 15 01:44:14.138: INFO: Trying to get logs from node node2 pod pod-5198098d-eef6-4689-a6d7-1b38b55f2186 container test-container: STEP: delete the pod May 15 01:44:14.152: INFO: Waiting for pod pod-5198098d-eef6-4689-a6d7-1b38b55f2186 to disappear May 15 01:44:14.153: INFO: Pod pod-5198098d-eef6-4689-a6d7-1b38b55f2186 no longer exists [AfterEach] [sig-storage] EmptyDir volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:44:14.154: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "emptydir-848" for this suite. • ------------------------------ {"msg":"PASSED [sig-storage] EmptyDir volumes when FSGroup is specified [LinuxOnly] [NodeFeature:FSGroup] files with FSGroup ownership should support (root,0644,tmpfs)","total":-1,"completed":2,"skipped":306,"failed":0} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] Dynamic Provisioning /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:43:58.422: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename volume-provisioning May 15 01:43:58.443: INFO: Found PodSecurityPolicies; testing pod creation to see if PodSecurityPolicy is enabled May 15 01:43:58.445: INFO: Error creating dryrun pod; assuming PodSecurityPolicy is disabled: admission webhook "cmk.intel.com" does not support dry run STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] Dynamic Provisioning /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/volume_provisioning.go:145 [It] should create and delete persistent volumes [fast] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/volume_provisioning.go:749 STEP: creating a Gluster DP server Pod STEP: locating the provisioner pod STEP: creating a StorageClass STEP: creating a claim object with a suffix for gluster dynamic provisioner May 15 01:44:10.470: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil STEP: creating a StorageClass volume-provisioning-1477-glusterdptest STEP: creating a claim May 15 01:44:10.480: INFO: Waiting up to 5m0s for PersistentVolumeClaims [pvc-5qjmv] to have phase Bound May 15 01:44:10.482: INFO: PersistentVolumeClaim pvc-5qjmv found but phase is Pending instead of Bound. May 15 01:44:12.488: INFO: PersistentVolumeClaim pvc-5qjmv found and phase=Bound (2.008150095s) STEP: checking the claim STEP: checking the PV STEP: deleting claim "volume-provisioning-1477"/"pvc-5qjmv" STEP: deleting the claim's PV "pvc-4840d26f-bf7a-4691-ad2d-94762f3276d3" May 15 01:44:12.495: INFO: Waiting up to 20m0s for PersistentVolume pvc-4840d26f-bf7a-4691-ad2d-94762f3276d3 to get deleted May 15 01:44:12.499: INFO: PersistentVolume pvc-4840d26f-bf7a-4691-ad2d-94762f3276d3 found and phase=Bound (3.41052ms) May 15 01:44:17.502: INFO: PersistentVolume pvc-4840d26f-bf7a-4691-ad2d-94762f3276d3 was removed May 15 01:44:17.502: INFO: deleting claim "volume-provisioning-1477"/"pvc-5qjmv" May 15 01:44:17.504: INFO: deleting storage class volume-provisioning-1477-glusterdptest [AfterEach] [sig-storage] Dynamic Provisioning /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:44:17.509: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "volume-provisioning-1477" for this suite. • [SLOW TEST:19.094 seconds] [sig-storage] Dynamic Provisioning /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [k8s.io] GlusterDynamicProvisioner /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:592 should create and delete persistent volumes [fast] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/volume_provisioning.go:749 ------------------------------ {"msg":"PASSED [sig-storage] Dynamic Provisioning [k8s.io] GlusterDynamicProvisioner should create and delete persistent volumes [fast]","total":-1,"completed":1,"skipped":44,"failed":0} SSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:44:11.698: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: dir-link-bindmounted] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes May 15 01:44:21.745: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir /tmp/local-volume-test-dece5cc4-aa85-4f88-b346-cddf9baae47c-backend && mount --bind /tmp/local-volume-test-dece5cc4-aa85-4f88-b346-cddf9baae47c-backend /tmp/local-volume-test-dece5cc4-aa85-4f88-b346-cddf9baae47c-backend && ln -s /tmp/local-volume-test-dece5cc4-aa85-4f88-b346-cddf9baae47c-backend /tmp/local-volume-test-dece5cc4-aa85-4f88-b346-cddf9baae47c] Namespace:persistent-local-volumes-test-5007 PodName:hostexec-node1-b5r72 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:44:21.745: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:44:22.742: INFO: Creating a PV followed by a PVC May 15 01:44:22.749: INFO: Waiting for PV local-pv4z7s2 to bind to PVC pvc-db2zd May 15 01:44:22.749: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-db2zd] to have phase Bound May 15 01:44:22.751: INFO: PersistentVolumeClaim pvc-db2zd found but phase is Pending instead of Bound. May 15 01:44:24.756: INFO: PersistentVolumeClaim pvc-db2zd found and phase=Bound (2.00698733s) May 15 01:44:24.756: INFO: Waiting up to 3m0s for PersistentVolume local-pv4z7s2 to have phase Bound May 15 01:44:24.758: INFO: PersistentVolume local-pv4z7s2 found and phase=Bound (2.172176ms) [BeforeEach] Set fsGroup for local volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:257 [It] should set different fsGroup for second pod if first pod is deleted /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:282 May 15 01:44:24.763: INFO: Disabled temporarily, reopen after #73168 is fixed [AfterEach] [Volume type: dir-link-bindmounted] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:44:24.765: INFO: Deleting PersistentVolumeClaim "pvc-db2zd" May 15 01:44:24.769: INFO: Deleting PersistentVolume "local-pv4z7s2" STEP: Removing the test directory May 15 01:44:24.773: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm /tmp/local-volume-test-dece5cc4-aa85-4f88-b346-cddf9baae47c && umount /tmp/local-volume-test-dece5cc4-aa85-4f88-b346-cddf9baae47c-backend && rm -r /tmp/local-volume-test-dece5cc4-aa85-4f88-b346-cddf9baae47c-backend] Namespace:persistent-local-volumes-test-5007 PodName:hostexec-node1-b5r72 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:44:24.773: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:44:24.906: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-5007" for this suite. S [SKIPPING] [13.216 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: dir-link-bindmounted] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 Set fsGroup for local volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:256 should set different fsGroup for second pod if first pod is deleted [It] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:282 Disabled temporarily, reopen after #73168 is fixed /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:283 ------------------------------ SSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] GCP Volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:44:24.948: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename gcp-volume STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] GCP Volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/volumes.go:69 May 15 01:44:24.966: INFO: Only supported for node OS distro [gci ubuntu custom] (not debian) [AfterEach] [sig-storage] GCP Volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:44:24.967: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "gcp-volume-670" for this suite. S [SKIPPING] in Spec Setup (BeforeEach) [0.027 seconds] [sig-storage] GCP Volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/volumes.go:61 GlusterFS [BeforeEach] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/volumes.go:129 should be mountable /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/volumes.go:130 Only supported for node OS distro [gci ubuntu custom] (not debian) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/volumes.go:70 ------------------------------ SSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PVC Protection /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:44:25.029: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename pvc-protection STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PVC Protection /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/pvc_protection.go:71 May 15 01:44:25.050: INFO: Waiting up to 30m0s for all (but 0) nodes to be schedulable STEP: Creating a PVC May 15 01:44:25.054: INFO: error finding default storageClass : No default storage class found [AfterEach] [sig-storage] PVC Protection /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:44:25.056: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "pvc-protection-3056" for this suite. [AfterEach] [sig-storage] PVC Protection /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/pvc_protection.go:106 S [SKIPPING] in Spec Setup (BeforeEach) [0.033 seconds] [sig-storage] PVC Protection /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 Verify "immediate" deletion of a PVC that is not in active use by a pod [BeforeEach] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/pvc_protection.go:112 error finding default storageClass : No default storage class found /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/pv/pv.go:825 ------------------------------ SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:44:17.563: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: tmpfs] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes STEP: Creating tmpfs mount point on node "node1" at path "/tmp/local-volume-test-55a1a32d-6ad3-4e38-add2-58a198ac75dc" May 15 01:44:29.607: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p "/tmp/local-volume-test-55a1a32d-6ad3-4e38-add2-58a198ac75dc" && mount -t tmpfs -o size=10m tmpfs-"/tmp/local-volume-test-55a1a32d-6ad3-4e38-add2-58a198ac75dc" "/tmp/local-volume-test-55a1a32d-6ad3-4e38-add2-58a198ac75dc"] Namespace:persistent-local-volumes-test-6218 PodName:hostexec-node1-dmkkc ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:44:29.607: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:44:29.931: INFO: Creating a PV followed by a PVC May 15 01:44:29.937: INFO: Waiting for PV local-pvf2lvj to bind to PVC pvc-mp9mc May 15 01:44:29.937: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-mp9mc] to have phase Bound May 15 01:44:29.940: INFO: PersistentVolumeClaim pvc-mp9mc found but phase is Pending instead of Bound. May 15 01:44:31.943: INFO: PersistentVolumeClaim pvc-mp9mc found and phase=Bound (2.005506722s) May 15 01:44:31.943: INFO: Waiting up to 3m0s for PersistentVolume local-pvf2lvj to have phase Bound May 15 01:44:31.945: INFO: PersistentVolume local-pvf2lvj found and phase=Bound (2.374695ms) [BeforeEach] Set fsGroup for local volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:257 [It] should set different fsGroup for second pod if first pod is deleted /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:282 May 15 01:44:31.950: INFO: Disabled temporarily, reopen after #73168 is fixed [AfterEach] [Volume type: tmpfs] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:44:31.952: INFO: Deleting PersistentVolumeClaim "pvc-mp9mc" May 15 01:44:31.957: INFO: Deleting PersistentVolume "local-pvf2lvj" STEP: Unmount tmpfs mount point on node "node1" at path "/tmp/local-volume-test-55a1a32d-6ad3-4e38-add2-58a198ac75dc" May 15 01:44:31.961: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c umount "/tmp/local-volume-test-55a1a32d-6ad3-4e38-add2-58a198ac75dc"] Namespace:persistent-local-volumes-test-6218 PodName:hostexec-node1-dmkkc ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:44:31.961: INFO: >>> kubeConfig: /root/.kube/config STEP: Removing the test directory May 15 01:44:32.170: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-55a1a32d-6ad3-4e38-add2-58a198ac75dc] Namespace:persistent-local-volumes-test-6218 PodName:hostexec-node1-dmkkc ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:44:32.170: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:44:32.382: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-6218" for this suite. S [SKIPPING] [14.827 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: tmpfs] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 Set fsGroup for local volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:256 should set different fsGroup for second pod if first pod is deleted [It] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:282 Disabled temporarily, reopen after #73168 is fixed /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:283 ------------------------------ SSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:44:25.187: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: dir] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes May 15 01:44:33.244: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-cd72aef7-fadb-4d91-b524-8729d14cf88b] Namespace:persistent-local-volumes-test-3367 PodName:hostexec-node1-5x9j8 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:44:33.244: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:44:33.370: INFO: Creating a PV followed by a PVC May 15 01:44:33.377: INFO: Waiting for PV local-pvms5d2 to bind to PVC pvc-xjqd5 May 15 01:44:33.377: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-xjqd5] to have phase Bound May 15 01:44:33.380: INFO: PersistentVolumeClaim pvc-xjqd5 found but phase is Pending instead of Bound. May 15 01:44:35.383: INFO: PersistentVolumeClaim pvc-xjqd5 found and phase=Bound (2.005810733s) May 15 01:44:35.383: INFO: Waiting up to 3m0s for PersistentVolume local-pvms5d2 to have phase Bound May 15 01:44:35.386: INFO: PersistentVolume local-pvms5d2 found and phase=Bound (2.395119ms) [BeforeEach] Set fsGroup for local volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:257 [It] should set different fsGroup for second pod if first pod is deleted /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:282 May 15 01:44:35.390: INFO: Disabled temporarily, reopen after #73168 is fixed [AfterEach] [Volume type: dir] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:44:35.391: INFO: Deleting PersistentVolumeClaim "pvc-xjqd5" May 15 01:44:35.395: INFO: Deleting PersistentVolume "local-pvms5d2" STEP: Removing the test directory May 15 01:44:35.399: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-cd72aef7-fadb-4d91-b524-8729d14cf88b] Namespace:persistent-local-volumes-test-3367 PodName:hostexec-node1-5x9j8 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:44:35.399: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:44:35.658: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-3367" for this suite. S [SKIPPING] [10.479 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: dir] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 Set fsGroup for local volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:256 should set different fsGroup for second pod if first pod is deleted [It] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:282 Disabled temporarily, reopen after #73168 is fixed /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:283 ------------------------------ SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:43:58.409: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test May 15 01:43:58.428: INFO: Found PodSecurityPolicies; testing pod creation to see if PodSecurityPolicy is enabled May 15 01:43:58.430: INFO: Error creating dryrun pod; assuming PodSecurityPolicy is disabled: admission webhook "cmk.intel.com" does not support dry run STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: blockfswithoutformat] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes STEP: Creating block device on node "node1" using path "/tmp/local-volume-test-2bae82b4-3bd6-4731-aa02-53284a851a86" May 15 01:44:02.458: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-2bae82b4-3bd6-4731-aa02-53284a851a86 && dd if=/dev/zero of=/tmp/local-volume-test-2bae82b4-3bd6-4731-aa02-53284a851a86/file bs=4096 count=5120 && losetup -f /tmp/local-volume-test-2bae82b4-3bd6-4731-aa02-53284a851a86/file] Namespace:persistent-local-volumes-test-5137 PodName:hostexec-node1-9zjjc ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:44:02.459: INFO: >>> kubeConfig: /root/.kube/config May 15 01:44:03.127: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c E2E_LOOP_DEV=$(losetup | grep /tmp/local-volume-test-2bae82b4-3bd6-4731-aa02-53284a851a86/file | awk '{ print $1 }') 2>&1 > /dev/null && echo ${E2E_LOOP_DEV}] Namespace:persistent-local-volumes-test-5137 PodName:hostexec-node1-9zjjc ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:44:03.128: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:44:03.249: INFO: Creating a PV followed by a PVC May 15 01:44:03.255: INFO: Waiting for PV local-pvwmcjc to bind to PVC pvc-568sd May 15 01:44:03.255: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-568sd] to have phase Bound May 15 01:44:03.258: INFO: PersistentVolumeClaim pvc-568sd found but phase is Pending instead of Bound. May 15 01:44:05.264: INFO: PersistentVolumeClaim pvc-568sd found and phase=Bound (2.00844538s) May 15 01:44:05.264: INFO: Waiting up to 3m0s for PersistentVolume local-pvwmcjc to have phase Bound May 15 01:44:05.267: INFO: PersistentVolume local-pvwmcjc found and phase=Bound (2.854867ms) [It] should be able to write from pod1 and read from pod2 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:251 STEP: Creating pod1 STEP: Creating a pod May 15 01:44:13.291: INFO: pod "pod-899b4246-5498-45aa-a5b2-0397a7417237" created on Node "node1" STEP: Writing in pod1 May 15 01:44:13.291: INFO: ExecWithOptions {Command:[/bin/sh -c mkdir -p /mnt/volume1; echo test-file-content > /mnt/volume1/test-file] Namespace:persistent-local-volumes-test-5137 PodName:pod-899b4246-5498-45aa-a5b2-0397a7417237 ContainerName:write-pod Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:false} May 15 01:44:13.291: INFO: >>> kubeConfig: /root/.kube/config May 15 01:44:13.434: INFO: podRWCmdExec cmd: "mkdir -p /mnt/volume1; echo test-file-content > /mnt/volume1/test-file", out: "", stderr: "", err: May 15 01:44:13.434: INFO: ExecWithOptions {Command:[/bin/sh -c cat /mnt/volume1/test-file] Namespace:persistent-local-volumes-test-5137 PodName:pod-899b4246-5498-45aa-a5b2-0397a7417237 ContainerName:write-pod Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:false} May 15 01:44:13.434: INFO: >>> kubeConfig: /root/.kube/config May 15 01:44:13.562: INFO: podRWCmdExec cmd: "cat /mnt/volume1/test-file", out: "test-file-content", stderr: "", err: STEP: Deleting pod1 STEP: Deleting pod pod-899b4246-5498-45aa-a5b2-0397a7417237 in namespace persistent-local-volumes-test-5137 STEP: Creating pod2 STEP: Creating a pod May 15 01:46:13.587: FAIL: Unexpected error: <*errors.errorString | 0xc00100cd90>: { s: "pod \"pod-416f0af8-32a5-4ef1-895d-dfd960c6365b\" is not Running: timed out waiting for the condition", } pod "pod-416f0af8-32a5-4ef1-895d-dfd960c6365b" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage.twoPodsReadWriteSerialTest(0xc002ea9760, 0xc00142c800, 0xc0038f7bf0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:780 +0x329 k8s.io/kubernetes/test/e2e/storage.glob..func20.2.5.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:252 +0x45 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000703800) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc000703800) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc000703800, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [Volume type: blockfswithoutformat] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:46:13.588: INFO: Deleting PersistentVolumeClaim "pvc-568sd" May 15 01:46:13.594: INFO: Deleting PersistentVolume "local-pvwmcjc" May 15 01:46:13.598: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c E2E_LOOP_DEV=$(losetup | grep /tmp/local-volume-test-2bae82b4-3bd6-4731-aa02-53284a851a86/file | awk '{ print $1 }') 2>&1 > /dev/null && echo ${E2E_LOOP_DEV}] Namespace:persistent-local-volumes-test-5137 PodName:hostexec-node1-9zjjc ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:46:13.598: INFO: >>> kubeConfig: /root/.kube/config STEP: Tear down block device "/dev/loop0" on node "node1" at path /tmp/local-volume-test-2bae82b4-3bd6-4731-aa02-53284a851a86/file May 15 01:46:13.804: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c losetup -d /dev/loop0] Namespace:persistent-local-volumes-test-5137 PodName:hostexec-node1-9zjjc ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:46:13.804: INFO: >>> kubeConfig: /root/.kube/config STEP: Removing the test directory /tmp/local-volume-test-2bae82b4-3bd6-4731-aa02-53284a851a86 May 15 01:46:13.912: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-2bae82b4-3bd6-4731-aa02-53284a851a86] Namespace:persistent-local-volumes-test-5137 PodName:hostexec-node1-9zjjc ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:46:13.912: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "persistent-local-volumes-test-5137". STEP: Found 20 events. May 15 01:46:14.046: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for hostexec-node1-9zjjc: { } Scheduled: Successfully assigned persistent-local-volumes-test-5137/hostexec-node1-9zjjc to node1 May 15 01:46:14.046: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-416f0af8-32a5-4ef1-895d-dfd960c6365b: { } Scheduled: Successfully assigned persistent-local-volumes-test-5137/pod-416f0af8-32a5-4ef1-895d-dfd960c6365b to node1 May 15 01:46:14.046: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-899b4246-5498-45aa-a5b2-0397a7417237: { } Scheduled: Successfully assigned persistent-local-volumes-test-5137/pod-899b4246-5498-45aa-a5b2-0397a7417237 to node1 May 15 01:46:14.046: INFO: At 2021-05-15 01:43:59 +0000 UTC - event for hostexec-node1-9zjjc: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 15 01:46:14.046: INFO: At 2021-05-15 01:44:00 +0000 UTC - event for hostexec-node1-9zjjc: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 1.138622651s May 15 01:46:14.046: INFO: At 2021-05-15 01:44:00 +0000 UTC - event for hostexec-node1-9zjjc: {kubelet node1} Created: Created container agnhost-container May 15 01:46:14.046: INFO: At 2021-05-15 01:44:01 +0000 UTC - event for hostexec-node1-9zjjc: {kubelet node1} Started: Started container agnhost-container May 15 01:46:14.046: INFO: At 2021-05-15 01:44:07 +0000 UTC - event for pod-899b4246-5498-45aa-a5b2-0397a7417237: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:46:14.047: INFO: At 2021-05-15 01:44:07 +0000 UTC - event for pod-899b4246-5498-45aa-a5b2-0397a7417237: {multus } AddedInterface: Add eth0 [10.244.3.118/24] May 15 01:46:14.047: INFO: At 2021-05-15 01:44:09 +0000 UTC - event for pod-899b4246-5498-45aa-a5b2-0397a7417237: {kubelet node1} Pulled: Successfully pulled image "docker.io/library/busybox:1.29" in 2.235143172s May 15 01:46:14.047: INFO: At 2021-05-15 01:44:10 +0000 UTC - event for pod-899b4246-5498-45aa-a5b2-0397a7417237: {kubelet node1} Started: Started container write-pod May 15 01:46:14.047: INFO: At 2021-05-15 01:44:10 +0000 UTC - event for pod-899b4246-5498-45aa-a5b2-0397a7417237: {kubelet node1} Created: Created container write-pod May 15 01:46:14.047: INFO: At 2021-05-15 01:44:13 +0000 UTC - event for pod-899b4246-5498-45aa-a5b2-0397a7417237: {kubelet node1} Killing: Stopping container write-pod May 15 01:46:14.047: INFO: At 2021-05-15 01:44:18 +0000 UTC - event for pod-416f0af8-32a5-4ef1-895d-dfd960c6365b: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:46:14.047: INFO: At 2021-05-15 01:44:18 +0000 UTC - event for pod-416f0af8-32a5-4ef1-895d-dfd960c6365b: {multus } AddedInterface: Add eth0 [10.244.3.122/24] May 15 01:46:14.047: INFO: At 2021-05-15 01:44:21 +0000 UTC - event for pod-416f0af8-32a5-4ef1-895d-dfd960c6365b: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 15 01:46:14.047: INFO: At 2021-05-15 01:44:21 +0000 UTC - event for pod-416f0af8-32a5-4ef1-895d-dfd960c6365b: {kubelet node1} Failed: Error: ImagePullBackOff May 15 01:46:14.047: INFO: At 2021-05-15 01:44:21 +0000 UTC - event for pod-416f0af8-32a5-4ef1-895d-dfd960c6365b: {kubelet node1} Failed: Error: ErrImagePull May 15 01:46:14.047: INFO: At 2021-05-15 01:44:21 +0000 UTC - event for pod-416f0af8-32a5-4ef1-895d-dfd960c6365b: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:46:14.047: INFO: At 2021-05-15 01:44:37 +0000 UTC - event for pod-416f0af8-32a5-4ef1-895d-dfd960c6365b: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:46:14.050: INFO: POD NODE PHASE GRACE CONDITIONS May 15 01:46:14.050: INFO: hostexec-node1-9zjjc node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:43:58 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:01 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:01 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:43:58 +0000 UTC }] May 15 01:46:14.050: INFO: pod-416f0af8-32a5-4ef1-895d-dfd960c6365b node1 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:13 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:13 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:13 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:13 +0000 UTC }] May 15 01:46:14.050: INFO: May 15 01:46:14.055: INFO: Logging node info for node master1 May 15 01:46:14.057: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 153753 0 2021-05-14 19:56:35 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:07 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:07 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:07 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:07 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[@ :],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:46:14.058: INFO: Logging kubelet events for node master1 May 15 01:46:14.062: INFO: Logging pods the kubelet thinks is on node master1 May 15 01:46:14.089: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.089: INFO: Container nfd-controller ready: true, restart count 0 May 15 01:46:14.089: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.089: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:46:14.089: INFO: kubernetes-metrics-scraper-678c97765c-fswrn started at 2021-05-15 00:18:49 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.089: INFO: Container kubernetes-metrics-scraper ready: true, restart count 0 May 15 01:46:14.089: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.089: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:46:14.089: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.089: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:14.089: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 15 01:46:14.089: INFO: Container docker-registry ready: true, restart count 0 May 15 01:46:14.089: INFO: Container nginx ready: true, restart count 0 May 15 01:46:14.089: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.089: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:46:14.089: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:14.089: INFO: Init container install-cni ready: true, restart count 0 May 15 01:46:14.089: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:46:14.089: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:46:14.089: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:14.089: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:14.089: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.089: INFO: Container kube-scheduler ready: true, restart count 0 W0515 01:46:14.102351 34 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:14.129: INFO: Latency metrics for node master1 May 15 01:46:14.129: INFO: Logging node info for node master2 May 15 01:46:14.131: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 153742 0 2021-05-14 19:57:04 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:05 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:05 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:05 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:05 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:46:14.132: INFO: Logging kubelet events for node master2 May 15 01:46:14.136: INFO: Logging pods the kubelet thinks is on node master2 May 15 01:46:14.151: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.151: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:46:14.151: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.151: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:14.151: INFO: coredns-7677f9bb54-96w24 started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.151: INFO: Container coredns ready: true, restart count 0 May 15 01:46:14.151: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.151: INFO: Container autoscaler ready: true, restart count 2 May 15 01:46:14.151: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:46:14.151: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:14.151: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:14.151: INFO: prometheus-operator-5bb8cb9d8f-fqb87 started at 2021-05-15 00:18:49 +0000 UTC (0+2 container statuses recorded) May 15 01:46:14.151: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:14.151: INFO: Container prometheus-operator ready: true, restart count 0 May 15 01:46:14.151: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.151: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:46:14.151: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.151: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:46:14.151: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.151: INFO: Container kube-scheduler ready: true, restart count 2 May 15 01:46:14.151: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:14.151: INFO: Init container install-cni ready: true, restart count 0 May 15 01:46:14.151: INFO: Container kube-flannel ready: true, restart count 1 W0515 01:46:14.163720 34 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:14.189: INFO: Latency metrics for node master2 May 15 01:46:14.189: INFO: Logging node info for node master3 May 15 01:46:14.192: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 153835 0 2021-05-14 19:57:14 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234767872 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324603904 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:13 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:13 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:13 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:13 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:46:14.192: INFO: Logging kubelet events for node master3 May 15 01:46:14.197: INFO: Logging pods the kubelet thinks is on node master3 May 15 01:46:14.211: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:46:14.211: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:14.211: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:14.211: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.211: INFO: Container kube-controller-manager ready: true, restart count 3 May 15 01:46:14.211: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.211: INFO: Container kube-scheduler ready: true, restart count 3 May 15 01:46:14.211: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.211: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:46:14.211: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.211: INFO: Container kube-proxy ready: true, restart count 1 May 15 01:46:14.211: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:14.211: INFO: Init container install-cni ready: true, restart count 0 May 15 01:46:14.211: INFO: Container kube-flannel ready: true, restart count 2 May 15 01:46:14.211: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.211: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:14.211: INFO: coredns-7677f9bb54-rpj8c started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.211: INFO: Container coredns ready: true, restart count 0 W0515 01:46:14.224830 34 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:14.256: INFO: Latency metrics for node master3 May 15 01:46:14.256: INFO: Logging node info for node node1 May 15 01:46:14.259: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 153831 0 2021-05-14 19:58:21 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-7734":"csi-mock-csi-mock-volumes-7734"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:00 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:44:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:46:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:13 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:13 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:13 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:13 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[@ :],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:46:14.259: INFO: Logging kubelet events for node node1 May 15 01:46:14.263: INFO: Logging pods the kubelet thinks is on node node1 May 15 01:46:14.287: INFO: pod-e1744dfa-1547-4ca5-acdf-8eb7113868b1 started at 2021-05-15 01:44:42 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.287: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:14.287: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.287: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:46:14.287: INFO: hostexec-node1-dzx4r started at 2021-05-15 01:43:58 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.287: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:14.287: INFO: hostexec-node1-9zjjc started at 2021-05-15 01:43:58 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.287: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:14.287: INFO: csi-mockplugin-0 started at 2021-05-15 01:43:59 +0000 UTC (0+3 container statuses recorded) May 15 01:46:14.287: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:46:14.287: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:46:14.287: INFO: Container mock ready: true, restart count 0 May 15 01:46:14.287: INFO: hostexec-node1-fbn4g started at 2021-05-15 01:44:09 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.287: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:14.287: INFO: pod-828316bc-b350-4ddb-9ce9-cec4d8c6cd45 started at 2021-05-15 01:44:13 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.287: INFO: Container write-pod ready: true, restart count 0 May 15 01:46:14.287: INFO: pod-38d6e6d5-eb01-4ca7-89a4-5da7e94c8500 started at 2021-05-15 01:44:25 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.287: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:14.287: INFO: pod-bfa4b2b7-b6e7-46d4-9222-2bd34e874439 started at 2021-05-15 01:44:21 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.287: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:14.287: INFO: pod-416f0af8-32a5-4ef1-895d-dfd960c6365b started at 2021-05-15 01:44:13 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.287: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:14.287: INFO: pod-dc18f44a-4eff-4c02-9f90-6144d79f277b started at 2021-05-15 01:44:29 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.287: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:14.287: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:14.287: INFO: Init container install-cni ready: true, restart count 0 May 15 01:46:14.287: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:46:14.287: INFO: node-feature-discovery-worker-bw8zg started at 2021-05-15 00:18:56 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.287: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:46:14.287: INFO: collectd-mrzps started at 2021-05-15 00:19:22 +0000 UTC (0+3 container statuses recorded) May 15 01:46:14.287: INFO: Container collectd ready: true, restart count 0 May 15 01:46:14.287: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:46:14.287: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:46:14.287: INFO: pod-141a54ce-270d-46f4-a49c-b05dd96fbcdd started at 2021-05-15 01:44:28 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.287: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:14.287: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.287: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:14.287: INFO: prometheus-k8s-0 started at 2021-05-15 00:19:01 +0000 UTC (0+5 container statuses recorded) May 15 01:46:14.287: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 15 01:46:14.287: INFO: Container grafana ready: true, restart count 0 May 15 01:46:14.287: INFO: Container prometheus ready: true, restart count 26 May 15 01:46:14.287: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 15 01:46:14.287: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 15 01:46:14.287: INFO: pod-1812e695-95fc-4df1-8d69-0fbfa00893b8 started at 2021-05-15 01:44:26 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.287: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:14.287: INFO: hostexec-node1-zpl46 started at 2021-05-15 01:44:13 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.287: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:14.287: INFO: node-exporter-flvqz started at 2021-05-15 00:18:55 +0000 UTC (0+2 container statuses recorded) May 15 01:46:14.287: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:14.287: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:14.287: INFO: hostexec-node1-xrzq8 started at 2021-05-15 01:44:14 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.287: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:14.287: INFO: csi-mockplugin-attacher-0 started at 2021-05-15 01:43:59 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.287: INFO: Container csi-attacher ready: true, restart count 0 May 15 01:46:14.287: INFO: cmk-4s6dm started at 2021-05-15 00:18:54 +0000 UTC (0+2 container statuses recorded) May 15 01:46:14.287: INFO: Container nodereport ready: true, restart count 0 May 15 01:46:14.287: INFO: Container reconcile ready: true, restart count 0 May 15 01:46:14.287: INFO: csi-mockplugin-resizer-0 started at 2021-05-15 01:43:59 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.287: INFO: Container csi-resizer ready: false, restart count 0 May 15 01:46:14.287: INFO: hostexec-node1-5mrx2 started at 2021-05-15 01:44:12 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.287: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:14.287: INFO: pod-1c1fa310-7f5b-4c20-8494-ee29c889ac5f started at 2021-05-15 01:44:17 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.287: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:14.287: INFO: hostexec-node1-dfp4d started at 2021-05-15 01:44:32 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.287: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:14.287: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.287: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:46:14.287: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4vxcc started at 2021-05-15 00:19:00 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.287: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:46:14.287: INFO: hostexec-node1-lr7tc started at 2021-05-15 01:43:59 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.287: INFO: Container agnhost-container ready: true, restart count 0 W0515 01:46:14.300655 34 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:14.360: INFO: Latency metrics for node node1 May 15 01:46:14.360: INFO: Logging node info for node node2 May 15 01:46:14.362: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 153757 0 2021-05-14 19:58:22 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-3052":"csi-mock-csi-mock-volumes-3052"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:46 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:44:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}},"f:status":{"f:volumesAttached":{}}}} {kubelet Update v1 2021-05-15 01:44:49 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}},"f:volumesInUse":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:09 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:09 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:09 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:09 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-3052^4],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-3052^4,DevicePath:,},},Config:nil,},} May 15 01:46:14.363: INFO: Logging kubelet events for node node2 May 15 01:46:14.367: INFO: Logging pods the kubelet thinks is on node node2 May 15 01:46:14.384: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 15 01:46:14.385: INFO: Container tas-controller ready: true, restart count 0 May 15 01:46:14.385: INFO: Container tas-extender ready: true, restart count 0 May 15 01:46:14.385: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 15 01:46:14.385: INFO: Container collectd ready: true, restart count 0 May 15 01:46:14.385: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:46:14.385: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:46:14.385: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.385: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:46:14.385: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.385: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:46:14.385: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 15 01:46:14.385: INFO: Container nodereport ready: true, restart count 0 May 15 01:46:14.385: INFO: Container reconcile ready: true, restart count 0 May 15 01:46:14.385: INFO: csi-mockplugin-attacher-0 started at 2021-05-15 01:44:36 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.385: INFO: Container csi-attacher ready: true, restart count 0 May 15 01:46:14.385: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.385: INFO: Container cmk-webhook ready: true, restart count 0 May 15 01:46:14.385: INFO: csi-mockplugin-0 started at 2021-05-15 01:44:36 +0000 UTC (0+3 container statuses recorded) May 15 01:46:14.385: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:46:14.385: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:46:14.385: INFO: Container mock ready: true, restart count 0 May 15 01:46:14.385: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 15 01:46:14.385: INFO: Container discover ready: false, restart count 0 May 15 01:46:14.385: INFO: Container init ready: false, restart count 0 May 15 01:46:14.385: INFO: Container install ready: false, restart count 0 May 15 01:46:14.385: INFO: pvc-volume-tester-8nftk started at 2021-05-15 01:44:47 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.385: INFO: Container volume-tester ready: true, restart count 0 May 15 01:46:14.385: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.385: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:46:14.385: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.385: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:14.385: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.385: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 15 01:46:14.385: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:46:14.385: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:46:14.385: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:14.385: INFO: Init container install-cni ready: true, restart count 1 May 15 01:46:14.385: INFO: Container kube-flannel ready: true, restart count 4 May 15 01:46:14.385: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:46:14.385: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:14.385: INFO: Container node-exporter ready: true, restart count 0 W0515 01:46:14.398500 34 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:14.436: INFO: Latency metrics for node node2 May 15 01:46:14.437: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-5137" for this suite. • Failure [136.036 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: blockfswithoutformat] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 Two pods mounting a local volume one after the other /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:250 should be able to write from pod1 and read from pod2 [It] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:251 May 15 01:46:13.587: Unexpected error: <*errors.errorString | 0xc00100cd90>: { s: "pod \"pod-416f0af8-32a5-4ef1-895d-dfd960c6365b\" is not Running: timed out waiting for the condition", } pod "pod-416f0af8-32a5-4ef1-895d-dfd960c6365b" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:780 ------------------------------ {"msg":"FAILED [sig-storage] PersistentVolumes-local [Volume type: blockfswithoutformat] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2","total":-1,"completed":0,"skipped":20,"failed":1,"failures":["[sig-storage] PersistentVolumes-local [Volume type: blockfswithoutformat] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2"]} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:43:58.524: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: tmpfs] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes STEP: Creating tmpfs mount point on node "node1" at path "/tmp/local-volume-test-3c1ee807-b60d-4831-875d-57e29f509eed" May 15 01:44:05.342: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p "/tmp/local-volume-test-3c1ee807-b60d-4831-875d-57e29f509eed" && mount -t tmpfs -o size=10m tmpfs-"/tmp/local-volume-test-3c1ee807-b60d-4831-875d-57e29f509eed" "/tmp/local-volume-test-3c1ee807-b60d-4831-875d-57e29f509eed"] Namespace:persistent-local-volumes-test-8048 PodName:hostexec-node1-lr7tc ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:44:05.342: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:44:05.486: INFO: Creating a PV followed by a PVC May 15 01:44:05.495: INFO: Waiting for PV local-pvrdt4x to bind to PVC pvc-7jrlv May 15 01:44:05.495: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-7jrlv] to have phase Bound May 15 01:44:05.498: INFO: PersistentVolumeClaim pvc-7jrlv found but phase is Pending instead of Bound. May 15 01:44:07.501: INFO: PersistentVolumeClaim pvc-7jrlv found and phase=Bound (2.005744324s) May 15 01:44:07.501: INFO: Waiting up to 3m0s for PersistentVolume local-pvrdt4x to have phase Bound May 15 01:44:07.503: INFO: PersistentVolume local-pvrdt4x found and phase=Bound (2.257311ms) [It] should be able to write from pod1 and read from pod2 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:251 STEP: Creating pod1 STEP: Creating a pod May 15 01:44:17.528: INFO: pod "pod-3ea98d65-9f70-44b7-bd3d-3f857d5b63cf" created on Node "node1" STEP: Writing in pod1 May 15 01:44:17.528: INFO: ExecWithOptions {Command:[/bin/sh -c mkdir -p /mnt/volume1; echo test-file-content > /mnt/volume1/test-file] Namespace:persistent-local-volumes-test-8048 PodName:pod-3ea98d65-9f70-44b7-bd3d-3f857d5b63cf ContainerName:write-pod Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:false} May 15 01:44:17.528: INFO: >>> kubeConfig: /root/.kube/config May 15 01:44:17.651: INFO: podRWCmdExec cmd: "mkdir -p /mnt/volume1; echo test-file-content > /mnt/volume1/test-file", out: "", stderr: "", err: May 15 01:44:17.651: INFO: ExecWithOptions {Command:[/bin/sh -c cat /mnt/volume1/test-file] Namespace:persistent-local-volumes-test-8048 PodName:pod-3ea98d65-9f70-44b7-bd3d-3f857d5b63cf ContainerName:write-pod Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:false} May 15 01:44:17.651: INFO: >>> kubeConfig: /root/.kube/config May 15 01:44:17.792: INFO: podRWCmdExec cmd: "cat /mnt/volume1/test-file", out: "test-file-content", stderr: "", err: STEP: Deleting pod1 STEP: Deleting pod pod-3ea98d65-9f70-44b7-bd3d-3f857d5b63cf in namespace persistent-local-volumes-test-8048 STEP: Creating pod2 STEP: Creating a pod May 15 01:46:17.818: FAIL: Unexpected error: <*errors.errorString | 0xc003f36090>: { s: "pod \"pod-1c1fa310-7f5b-4c20-8494-ee29c889ac5f\" is not Running: timed out waiting for the condition", } pod "pod-1c1fa310-7f5b-4c20-8494-ee29c889ac5f" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage.twoPodsReadWriteSerialTest(0xc000df1e40, 0xc001f39580, 0xc003f54420) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:780 +0x329 k8s.io/kubernetes/test/e2e/storage.glob..func20.2.5.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:252 +0x45 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000683980) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc000683980) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc000683980, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [Volume type: tmpfs] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:46:17.819: INFO: Deleting PersistentVolumeClaim "pvc-7jrlv" May 15 01:46:17.824: INFO: Deleting PersistentVolume "local-pvrdt4x" STEP: Unmount tmpfs mount point on node "node1" at path "/tmp/local-volume-test-3c1ee807-b60d-4831-875d-57e29f509eed" May 15 01:46:17.829: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c umount "/tmp/local-volume-test-3c1ee807-b60d-4831-875d-57e29f509eed"] Namespace:persistent-local-volumes-test-8048 PodName:hostexec-node1-lr7tc ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:46:17.829: INFO: >>> kubeConfig: /root/.kube/config STEP: Removing the test directory May 15 01:46:17.952: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-3c1ee807-b60d-4831-875d-57e29f509eed] Namespace:persistent-local-volumes-test-8048 PodName:hostexec-node1-lr7tc ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:46:17.952: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "persistent-local-volumes-test-8048". STEP: Found 19 events. May 15 01:46:18.072: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for hostexec-node1-lr7tc: { } Scheduled: Successfully assigned persistent-local-volumes-test-8048/hostexec-node1-lr7tc to node1 May 15 01:46:18.073: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-1c1fa310-7f5b-4c20-8494-ee29c889ac5f: { } Scheduled: Successfully assigned persistent-local-volumes-test-8048/pod-1c1fa310-7f5b-4c20-8494-ee29c889ac5f to node1 May 15 01:46:18.073: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-3ea98d65-9f70-44b7-bd3d-3f857d5b63cf: { } Scheduled: Successfully assigned persistent-local-volumes-test-8048/pod-3ea98d65-9f70-44b7-bd3d-3f857d5b63cf to node1 May 15 01:46:18.073: INFO: At 2021-05-15 01:44:00 +0000 UTC - event for hostexec-node1-lr7tc: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 15 01:46:18.073: INFO: At 2021-05-15 01:44:01 +0000 UTC - event for hostexec-node1-lr7tc: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 1.411162721s May 15 01:46:18.073: INFO: At 2021-05-15 01:44:01 +0000 UTC - event for hostexec-node1-lr7tc: {kubelet node1} Created: Created container agnhost-container May 15 01:46:18.073: INFO: At 2021-05-15 01:44:03 +0000 UTC - event for hostexec-node1-lr7tc: {kubelet node1} Started: Started container agnhost-container May 15 01:46:18.073: INFO: At 2021-05-15 01:44:10 +0000 UTC - event for pod-3ea98d65-9f70-44b7-bd3d-3f857d5b63cf: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:46:18.073: INFO: At 2021-05-15 01:44:10 +0000 UTC - event for pod-3ea98d65-9f70-44b7-bd3d-3f857d5b63cf: {multus } AddedInterface: Add eth0 [10.244.3.120/24] May 15 01:46:18.073: INFO: At 2021-05-15 01:44:13 +0000 UTC - event for pod-3ea98d65-9f70-44b7-bd3d-3f857d5b63cf: {kubelet node1} Started: Started container write-pod May 15 01:46:18.073: INFO: At 2021-05-15 01:44:13 +0000 UTC - event for pod-3ea98d65-9f70-44b7-bd3d-3f857d5b63cf: {kubelet node1} Created: Created container write-pod May 15 01:46:18.073: INFO: At 2021-05-15 01:44:13 +0000 UTC - event for pod-3ea98d65-9f70-44b7-bd3d-3f857d5b63cf: {kubelet node1} Pulled: Successfully pulled image "docker.io/library/busybox:1.29" in 2.868354061s May 15 01:46:18.073: INFO: At 2021-05-15 01:44:17 +0000 UTC - event for pod-3ea98d65-9f70-44b7-bd3d-3f857d5b63cf: {kubelet node1} Killing: Stopping container write-pod May 15 01:46:18.073: INFO: At 2021-05-15 01:44:21 +0000 UTC - event for pod-1c1fa310-7f5b-4c20-8494-ee29c889ac5f: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:46:18.073: INFO: At 2021-05-15 01:44:21 +0000 UTC - event for pod-1c1fa310-7f5b-4c20-8494-ee29c889ac5f: {multus } AddedInterface: Add eth0 [10.244.3.123/24] May 15 01:46:18.073: INFO: At 2021-05-15 01:44:23 +0000 UTC - event for pod-1c1fa310-7f5b-4c20-8494-ee29c889ac5f: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:46:18.073: INFO: At 2021-05-15 01:44:23 +0000 UTC - event for pod-1c1fa310-7f5b-4c20-8494-ee29c889ac5f: {kubelet node1} Failed: Error: ErrImagePull May 15 01:46:18.073: INFO: At 2021-05-15 01:44:25 +0000 UTC - event for pod-1c1fa310-7f5b-4c20-8494-ee29c889ac5f: {kubelet node1} Failed: Error: ImagePullBackOff May 15 01:46:18.073: INFO: At 2021-05-15 01:44:25 +0000 UTC - event for pod-1c1fa310-7f5b-4c20-8494-ee29c889ac5f: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 15 01:46:18.075: INFO: POD NODE PHASE GRACE CONDITIONS May 15 01:46:18.075: INFO: hostexec-node1-lr7tc node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:43:59 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:04 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:04 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:43:59 +0000 UTC }] May 15 01:46:18.075: INFO: pod-1c1fa310-7f5b-4c20-8494-ee29c889ac5f node1 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:17 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:17 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:17 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:17 +0000 UTC }] May 15 01:46:18.075: INFO: May 15 01:46:18.080: INFO: Logging node info for node master1 May 15 01:46:18.082: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 153911 0 2021-05-14 19:56:35 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:17 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:17 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:17 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:17 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[@ :],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:46:18.082: INFO: Logging kubelet events for node master1 May 15 01:46:18.087: INFO: Logging pods the kubelet thinks is on node master1 May 15 01:46:18.107: INFO: kubernetes-metrics-scraper-678c97765c-fswrn started at 2021-05-15 00:18:49 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.107: INFO: Container kubernetes-metrics-scraper ready: true, restart count 0 May 15 01:46:18.107: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.107: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:46:18.107: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.107: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:18.107: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 15 01:46:18.107: INFO: Container docker-registry ready: true, restart count 0 May 15 01:46:18.107: INFO: Container nginx ready: true, restart count 0 May 15 01:46:18.107: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.107: INFO: Container nfd-controller ready: true, restart count 0 May 15 01:46:18.107: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.107: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:46:18.107: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:18.107: INFO: Init container install-cni ready: true, restart count 0 May 15 01:46:18.107: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:46:18.107: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:46:18.107: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:18.107: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:18.107: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.107: INFO: Container kube-scheduler ready: true, restart count 0 May 15 01:46:18.107: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.107: INFO: Container kube-controller-manager ready: true, restart count 2 W0515 01:46:18.120101 35 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:18.145: INFO: Latency metrics for node master1 May 15 01:46:18.145: INFO: Logging node info for node master2 May 15 01:46:18.148: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 153862 0 2021-05-14 19:57:04 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:15 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:15 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:15 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:15 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:46:18.148: INFO: Logging kubelet events for node master2 May 15 01:46:18.152: INFO: Logging pods the kubelet thinks is on node master2 May 15 01:46:18.160: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:46:18.160: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:18.160: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:18.160: INFO: prometheus-operator-5bb8cb9d8f-fqb87 started at 2021-05-15 00:18:49 +0000 UTC (0+2 container statuses recorded) May 15 01:46:18.160: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:18.160: INFO: Container prometheus-operator ready: true, restart count 0 May 15 01:46:18.160: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.160: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:46:18.160: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.160: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:46:18.160: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.160: INFO: Container kube-scheduler ready: true, restart count 2 May 15 01:46:18.161: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:18.161: INFO: Init container install-cni ready: true, restart count 0 May 15 01:46:18.161: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:46:18.161: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.161: INFO: Container autoscaler ready: true, restart count 2 May 15 01:46:18.161: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.161: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:46:18.161: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.161: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:18.161: INFO: coredns-7677f9bb54-96w24 started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.161: INFO: Container coredns ready: true, restart count 0 W0515 01:46:18.172959 35 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:18.205: INFO: Latency metrics for node master2 May 15 01:46:18.205: INFO: Logging node info for node master3 May 15 01:46:18.209: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 153835 0 2021-05-14 19:57:14 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234767872 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324603904 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:13 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:13 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:13 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:13 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:46:18.209: INFO: Logging kubelet events for node master3 May 15 01:46:18.214: INFO: Logging pods the kubelet thinks is on node master3 May 15 01:46:18.221: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.221: INFO: Container kube-scheduler ready: true, restart count 3 May 15 01:46:18.221: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.221: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:46:18.221: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.221: INFO: Container kube-proxy ready: true, restart count 1 May 15 01:46:18.221: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:18.221: INFO: Init container install-cni ready: true, restart count 0 May 15 01:46:18.221: INFO: Container kube-flannel ready: true, restart count 2 May 15 01:46:18.221: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.221: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:18.221: INFO: coredns-7677f9bb54-rpj8c started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.221: INFO: Container coredns ready: true, restart count 0 May 15 01:46:18.221: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:46:18.221: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:18.221: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:18.221: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.221: INFO: Container kube-controller-manager ready: true, restart count 3 W0515 01:46:18.232373 35 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:18.255: INFO: Latency metrics for node master3 May 15 01:46:18.255: INFO: Logging node info for node node1 May 15 01:46:18.258: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 153831 0 2021-05-14 19:58:21 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-7734":"csi-mock-csi-mock-volumes-7734"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:00 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:44:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:46:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:13 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:13 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:13 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:13 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[@ :],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:46:18.259: INFO: Logging kubelet events for node node1 May 15 01:46:18.263: INFO: Logging pods the kubelet thinks is on node node1 May 15 01:46:18.284: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.284: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:46:18.284: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4vxcc started at 2021-05-15 00:19:00 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.284: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:46:18.284: INFO: hostexec-node1-lr7tc started at 2021-05-15 01:43:59 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.284: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:18.284: INFO: hostexec-node1-fbn4g started at 2021-05-15 01:44:09 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.284: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:18.284: INFO: pod-828316bc-b350-4ddb-9ce9-cec4d8c6cd45 started at 2021-05-15 01:44:13 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.284: INFO: Container write-pod ready: true, restart count 0 May 15 01:46:18.284: INFO: pod-38d6e6d5-eb01-4ca7-89a4-5da7e94c8500 started at 2021-05-15 01:44:25 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.284: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:18.284: INFO: pod-e1744dfa-1547-4ca5-acdf-8eb7113868b1 started at 2021-05-15 01:44:42 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.284: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:18.284: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.284: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:46:18.284: INFO: hostexec-node1-dzx4r started at 2021-05-15 01:43:58 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.284: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:18.284: INFO: hostexec-node1-9zjjc started at 2021-05-15 01:43:58 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.284: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:18.284: INFO: csi-mockplugin-0 started at 2021-05-15 01:43:59 +0000 UTC (0+3 container statuses recorded) May 15 01:46:18.284: INFO: Container csi-provisioner ready: false, restart count 0 May 15 01:46:18.284: INFO: Container driver-registrar ready: false, restart count 0 May 15 01:46:18.284: INFO: Container mock ready: false, restart count 0 May 15 01:46:18.284: INFO: pod-bfa4b2b7-b6e7-46d4-9222-2bd34e874439 started at 2021-05-15 01:44:21 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.284: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:18.284: INFO: pod-416f0af8-32a5-4ef1-895d-dfd960c6365b started at 2021-05-15 01:44:13 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.284: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:18.284: INFO: pod-dc18f44a-4eff-4c02-9f90-6144d79f277b started at 2021-05-15 01:44:29 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.284: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:18.284: INFO: pod-141a54ce-270d-46f4-a49c-b05dd96fbcdd started at 2021-05-15 01:44:28 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.284: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:18.284: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:18.284: INFO: Init container install-cni ready: true, restart count 0 May 15 01:46:18.284: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:46:18.284: INFO: node-feature-discovery-worker-bw8zg started at 2021-05-15 00:18:56 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.284: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:46:18.284: INFO: collectd-mrzps started at 2021-05-15 00:19:22 +0000 UTC (0+3 container statuses recorded) May 15 01:46:18.284: INFO: Container collectd ready: true, restart count 0 May 15 01:46:18.284: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:46:18.284: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:46:18.284: INFO: hostexec-node1-lmdw4 started at 2021-05-15 01:46:14 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.284: INFO: Container agnhost-container ready: false, restart count 0 May 15 01:46:18.284: INFO: hostexec-node1-zpl46 started at 2021-05-15 01:44:13 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.284: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:18.284: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.284: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:18.284: INFO: prometheus-k8s-0 started at 2021-05-15 00:19:01 +0000 UTC (0+5 container statuses recorded) May 15 01:46:18.284: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 15 01:46:18.284: INFO: Container grafana ready: true, restart count 0 May 15 01:46:18.284: INFO: Container prometheus ready: true, restart count 26 May 15 01:46:18.284: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 15 01:46:18.284: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 15 01:46:18.284: INFO: pod-1812e695-95fc-4df1-8d69-0fbfa00893b8 started at 2021-05-15 01:44:26 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.284: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:18.284: INFO: node-exporter-flvqz started at 2021-05-15 00:18:55 +0000 UTC (0+2 container statuses recorded) May 15 01:46:18.284: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:18.284: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:18.284: INFO: hostexec-node1-xrzq8 started at 2021-05-15 01:44:14 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.284: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:18.284: INFO: pod-1c1fa310-7f5b-4c20-8494-ee29c889ac5f started at 2021-05-15 01:44:17 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.284: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:18.284: INFO: csi-mockplugin-attacher-0 started at 2021-05-15 01:43:59 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.284: INFO: Container csi-attacher ready: false, restart count 0 May 15 01:46:18.284: INFO: cmk-4s6dm started at 2021-05-15 00:18:54 +0000 UTC (0+2 container statuses recorded) May 15 01:46:18.284: INFO: Container nodereport ready: true, restart count 0 May 15 01:46:18.284: INFO: Container reconcile ready: true, restart count 0 May 15 01:46:18.284: INFO: hostexec-node1-5mrx2 started at 2021-05-15 01:44:12 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.284: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:18.284: INFO: hostexec-node1-dfp4d started at 2021-05-15 01:44:32 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.284: INFO: Container agnhost-container ready: true, restart count 0 W0515 01:46:18.298456 35 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:18.342: INFO: Latency metrics for node node1 May 15 01:46:18.342: INFO: Logging node info for node node2 May 15 01:46:18.346: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 153757 0 2021-05-14 19:58:22 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-3052":"csi-mock-csi-mock-volumes-3052"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:46 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:44:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}},"f:status":{"f:volumesAttached":{}}}} {kubelet Update v1 2021-05-15 01:44:49 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}},"f:volumesInUse":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:09 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:09 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:09 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:09 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-3052^4],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-3052^4,DevicePath:,},},Config:nil,},} May 15 01:46:18.346: INFO: Logging kubelet events for node node2 May 15 01:46:18.350: INFO: Logging pods the kubelet thinks is on node node2 May 15 01:46:18.362: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.362: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:46:18.362: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 15 01:46:18.362: INFO: Container nodereport ready: true, restart count 0 May 15 01:46:18.362: INFO: Container reconcile ready: true, restart count 0 May 15 01:46:18.362: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.362: INFO: Container cmk-webhook ready: true, restart count 0 May 15 01:46:18.362: INFO: csi-mockplugin-0 started at 2021-05-15 01:44:36 +0000 UTC (0+3 container statuses recorded) May 15 01:46:18.362: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:46:18.362: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:46:18.362: INFO: Container mock ready: true, restart count 0 May 15 01:46:18.362: INFO: csi-mockplugin-attacher-0 started at 2021-05-15 01:44:36 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.362: INFO: Container csi-attacher ready: true, restart count 0 May 15 01:46:18.362: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 15 01:46:18.362: INFO: Container discover ready: false, restart count 0 May 15 01:46:18.362: INFO: Container init ready: false, restart count 0 May 15 01:46:18.362: INFO: Container install ready: false, restart count 0 May 15 01:46:18.362: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.362: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:46:18.362: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.362: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:18.362: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.362: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 15 01:46:18.362: INFO: pvc-volume-tester-8nftk started at 2021-05-15 01:44:47 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.362: INFO: Container volume-tester ready: true, restart count 0 May 15 01:46:18.362: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.362: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:46:18.362: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:18.362: INFO: Init container install-cni ready: true, restart count 1 May 15 01:46:18.362: INFO: Container kube-flannel ready: true, restart count 4 May 15 01:46:18.362: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:46:18.363: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:18.363: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:18.363: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 15 01:46:18.363: INFO: Container tas-controller ready: true, restart count 0 May 15 01:46:18.363: INFO: Container tas-extender ready: true, restart count 0 May 15 01:46:18.363: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 15 01:46:18.363: INFO: Container collectd ready: true, restart count 0 May 15 01:46:18.363: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:46:18.363: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:46:18.363: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 15 01:46:18.363: INFO: Container kube-sriovdp ready: true, restart count 0 W0515 01:46:18.373334 35 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:18.422: INFO: Latency metrics for node node2 May 15 01:46:18.422: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-8048" for this suite. • Failure [139.906 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: tmpfs] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 Two pods mounting a local volume one after the other /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:250 should be able to write from pod1 and read from pod2 [It] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:251 May 15 01:46:17.818: Unexpected error: <*errors.errorString | 0xc003f36090>: { s: "pod \"pod-1c1fa310-7f5b-4c20-8494-ee29c889ac5f\" is not Running: timed out waiting for the condition", } pod "pod-1c1fa310-7f5b-4c20-8494-ee29c889ac5f" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:780 ------------------------------ {"msg":"FAILED [sig-storage] PersistentVolumes-local [Volume type: tmpfs] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2","total":-1,"completed":0,"skipped":62,"failed":1,"failures":["[sig-storage] PersistentVolumes-local [Volume type: tmpfs] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2"]} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] Dynamic Provisioning /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:46:18.840: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename volume-provisioning STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] Dynamic Provisioning /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/volume_provisioning.go:145 [It] should report an error and create no PV /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/volume_provisioning.go:778 May 15 01:46:18.861: INFO: Only supported for providers [aws] (not skeleton) [AfterEach] [sig-storage] Dynamic Provisioning /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:46:18.862: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "volume-provisioning-8110" for this suite. S [SKIPPING] [0.030 seconds] [sig-storage] Dynamic Provisioning /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 Invalid AWS KMS key /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/volume_provisioning.go:777 should report an error and create no PV [It] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/volume_provisioning.go:778 Only supported for providers [aws] (not skeleton) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/volume_provisioning.go:779 ------------------------------ SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:44:09.515: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: dir-link] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes May 15 01:44:19.565: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir /tmp/local-volume-test-355f1869-c80b-4150-bfdf-fc91c5b054e3-backend && ln -s /tmp/local-volume-test-355f1869-c80b-4150-bfdf-fc91c5b054e3-backend /tmp/local-volume-test-355f1869-c80b-4150-bfdf-fc91c5b054e3] Namespace:persistent-local-volumes-test-6549 PodName:hostexec-node1-fbn4g ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:44:19.565: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:44:19.678: INFO: Creating a PV followed by a PVC May 15 01:44:19.685: INFO: Waiting for PV local-pvktzjj to bind to PVC pvc-lq42h May 15 01:44:19.685: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-lq42h] to have phase Bound May 15 01:44:19.688: INFO: PersistentVolumeClaim pvc-lq42h found but phase is Pending instead of Bound. May 15 01:44:21.690: INFO: PersistentVolumeClaim pvc-lq42h found and phase=Bound (2.004920794s) May 15 01:44:21.690: INFO: Waiting up to 3m0s for PersistentVolume local-pvktzjj to have phase Bound May 15 01:44:21.693: INFO: PersistentVolume local-pvktzjj found and phase=Bound (2.124347ms) [It] should be able to write from pod1 and read from pod2 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:245 STEP: Creating pod1 to write to the PV STEP: Creating a pod May 15 01:46:21.718: FAIL: Unexpected error: <*errors.errorString | 0xc000ab9180>: { s: "pod \"pod-bfa4b2b7-b6e7-46d4-9222-2bd34e874439\" is not Running: timed out waiting for the condition", } pod "pod-bfa4b2b7-b6e7-46d4-9222-2bd34e874439" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage.twoPodsReadWriteTest(0xc002176160, 0xc003c2ab00, 0xc003368cf0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:727 +0xaf k8s.io/kubernetes/test/e2e/storage.glob..func20.2.4.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:246 +0x45 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc003230480) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc003230480) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc003230480, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [Volume type: dir-link] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:46:21.720: INFO: Deleting PersistentVolumeClaim "pvc-lq42h" May 15 01:46:21.724: INFO: Deleting PersistentVolume "local-pvktzjj" STEP: Removing the test directory May 15 01:46:21.728: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-355f1869-c80b-4150-bfdf-fc91c5b054e3 && rm -r /tmp/local-volume-test-355f1869-c80b-4150-bfdf-fc91c5b054e3-backend] Namespace:persistent-local-volumes-test-6549 PodName:hostexec-node1-fbn4g ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:46:21.728: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "persistent-local-volumes-test-6549". STEP: Found 13 events. May 15 01:46:21.851: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for hostexec-node1-fbn4g: { } Scheduled: Successfully assigned persistent-local-volumes-test-6549/hostexec-node1-fbn4g to node1 May 15 01:46:21.851: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-bfa4b2b7-b6e7-46d4-9222-2bd34e874439: { } Scheduled: Successfully assigned persistent-local-volumes-test-6549/pod-bfa4b2b7-b6e7-46d4-9222-2bd34e874439 to node1 May 15 01:46:21.851: INFO: At 2021-05-15 01:44:10 +0000 UTC - event for hostexec-node1-fbn4g: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 15 01:46:21.851: INFO: At 2021-05-15 01:44:13 +0000 UTC - event for hostexec-node1-fbn4g: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 3.137272242s May 15 01:46:21.851: INFO: At 2021-05-15 01:44:13 +0000 UTC - event for hostexec-node1-fbn4g: {kubelet node1} Created: Created container agnhost-container May 15 01:46:21.851: INFO: At 2021-05-15 01:44:13 +0000 UTC - event for hostexec-node1-fbn4g: {kubelet node1} Started: Started container agnhost-container May 15 01:46:21.851: INFO: At 2021-05-15 01:44:25 +0000 UTC - event for pod-bfa4b2b7-b6e7-46d4-9222-2bd34e874439: {multus } AddedInterface: Add eth0 [10.244.3.124/24] May 15 01:46:21.851: INFO: At 2021-05-15 01:44:26 +0000 UTC - event for pod-bfa4b2b7-b6e7-46d4-9222-2bd34e874439: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:46:21.851: INFO: At 2021-05-15 01:44:27 +0000 UTC - event for pod-bfa4b2b7-b6e7-46d4-9222-2bd34e874439: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:46:21.851: INFO: At 2021-05-15 01:44:27 +0000 UTC - event for pod-bfa4b2b7-b6e7-46d4-9222-2bd34e874439: {kubelet node1} Failed: Error: ErrImagePull May 15 01:46:21.851: INFO: At 2021-05-15 01:44:28 +0000 UTC - event for pod-bfa4b2b7-b6e7-46d4-9222-2bd34e874439: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 15 01:46:21.851: INFO: At 2021-05-15 01:44:28 +0000 UTC - event for pod-bfa4b2b7-b6e7-46d4-9222-2bd34e874439: {kubelet node1} Failed: Error: ImagePullBackOff May 15 01:46:21.851: INFO: At 2021-05-15 01:44:41 +0000 UTC - event for pod-bfa4b2b7-b6e7-46d4-9222-2bd34e874439: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:46:21.853: INFO: POD NODE PHASE GRACE CONDITIONS May 15 01:46:21.853: INFO: hostexec-node1-fbn4g node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:09 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:14 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:14 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:09 +0000 UTC }] May 15 01:46:21.853: INFO: pod-bfa4b2b7-b6e7-46d4-9222-2bd34e874439 node1 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:21 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:21 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:21 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:21 +0000 UTC }] May 15 01:46:21.853: INFO: May 15 01:46:21.857: INFO: Logging node info for node master1 May 15 01:46:21.860: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 153911 0 2021-05-14 19:56:35 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:17 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:17 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:17 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:17 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[@ :],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:46:21.860: INFO: Logging kubelet events for node master1 May 15 01:46:21.865: INFO: Logging pods the kubelet thinks is on node master1 May 15 01:46:21.887: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:46:21.887: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:46:21.887: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:21.887: INFO: Init container install-cni ready: true, restart count 0 May 15 01:46:21.887: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:46:21.887: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:46:21.887: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:21.887: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:21.887: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:21.887: INFO: Container kube-scheduler ready: true, restart count 0 May 15 01:46:21.887: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 15 01:46:21.887: INFO: Container docker-registry ready: true, restart count 0 May 15 01:46:21.887: INFO: Container nginx ready: true, restart count 0 May 15 01:46:21.887: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 15 01:46:21.887: INFO: Container nfd-controller ready: true, restart count 0 May 15 01:46:21.887: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:21.887: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:46:21.887: INFO: kubernetes-metrics-scraper-678c97765c-fswrn started at 2021-05-15 00:18:49 +0000 UTC (0+1 container statuses recorded) May 15 01:46:21.887: INFO: Container kubernetes-metrics-scraper ready: true, restart count 0 May 15 01:46:21.887: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:21.887: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:46:21.887: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:21.887: INFO: Container kube-multus ready: true, restart count 1 W0515 01:46:21.898898 32 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:21.924: INFO: Latency metrics for node master1 May 15 01:46:21.924: INFO: Logging node info for node master2 May 15 01:46:21.926: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 153862 0 2021-05-14 19:57:04 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:15 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:15 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:15 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:15 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:46:21.927: INFO: Logging kubelet events for node master2 May 15 01:46:21.931: INFO: Logging pods the kubelet thinks is on node master2 May 15 01:46:21.939: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:21.939: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:46:21.939: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:21.939: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:21.939: INFO: coredns-7677f9bb54-96w24 started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:46:21.939: INFO: Container coredns ready: true, restart count 0 May 15 01:46:21.939: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:46:21.939: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:21.939: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:21.939: INFO: prometheus-operator-5bb8cb9d8f-fqb87 started at 2021-05-15 00:18:49 +0000 UTC (0+2 container statuses recorded) May 15 01:46:21.939: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:21.939: INFO: Container prometheus-operator ready: true, restart count 0 May 15 01:46:21.939: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:46:21.939: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:46:21.939: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:46:21.939: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:46:21.939: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:21.939: INFO: Container kube-scheduler ready: true, restart count 2 May 15 01:46:21.939: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:21.939: INFO: Init container install-cni ready: true, restart count 0 May 15 01:46:21.939: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:46:21.939: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 15 01:46:21.939: INFO: Container autoscaler ready: true, restart count 2 W0515 01:46:21.951136 32 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:21.975: INFO: Latency metrics for node master2 May 15 01:46:21.975: INFO: Logging node info for node master3 May 15 01:46:21.978: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 153835 0 2021-05-14 19:57:14 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234767872 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324603904 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:13 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:13 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:13 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:13 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:46:21.978: INFO: Logging kubelet events for node master3 May 15 01:46:21.982: INFO: Logging pods the kubelet thinks is on node master3 May 15 01:46:21.990: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:21.990: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:21.990: INFO: coredns-7677f9bb54-rpj8c started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:46:21.990: INFO: Container coredns ready: true, restart count 0 May 15 01:46:21.990: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:46:21.990: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:21.990: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:21.990: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 15 01:46:21.990: INFO: Container kube-controller-manager ready: true, restart count 3 May 15 01:46:21.990: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:21.990: INFO: Container kube-scheduler ready: true, restart count 3 May 15 01:46:21.990: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:21.990: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:46:21.990: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:21.990: INFO: Container kube-proxy ready: true, restart count 1 May 15 01:46:21.990: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:21.990: INFO: Init container install-cni ready: true, restart count 0 May 15 01:46:21.990: INFO: Container kube-flannel ready: true, restart count 2 W0515 01:46:22.002803 32 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:22.027: INFO: Latency metrics for node master3 May 15 01:46:22.027: INFO: Logging node info for node node1 May 15 01:46:22.031: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 153831 0 2021-05-14 19:58:21 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-7734":"csi-mock-csi-mock-volumes-7734"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:00 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:44:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:46:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:13 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:13 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:13 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:13 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[@ :],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:46:22.031: INFO: Logging kubelet events for node node1 May 15 01:46:22.036: INFO: Logging pods the kubelet thinks is on node node1 May 15 01:46:22.052: INFO: hostexec-node1-dfp4d started at 2021-05-15 01:44:32 +0000 UTC (0+1 container statuses recorded) May 15 01:46:22.052: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:22.052: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:22.052: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:46:22.052: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4vxcc started at 2021-05-15 00:19:00 +0000 UTC (0+1 container statuses recorded) May 15 01:46:22.052: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:46:22.052: INFO: hostexec-node1-lr7tc started at 2021-05-15 01:43:59 +0000 UTC (0+1 container statuses recorded) May 15 01:46:22.052: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:22.052: INFO: pod-38d6e6d5-eb01-4ca7-89a4-5da7e94c8500 started at 2021-05-15 01:44:25 +0000 UTC (0+1 container statuses recorded) May 15 01:46:22.052: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:22.052: INFO: pod-e1744dfa-1547-4ca5-acdf-8eb7113868b1 started at 2021-05-15 01:44:42 +0000 UTC (0+1 container statuses recorded) May 15 01:46:22.052: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:22.052: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:46:22.052: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:46:22.052: INFO: hostexec-node1-dzx4r started at 2021-05-15 01:43:58 +0000 UTC (0+1 container statuses recorded) May 15 01:46:22.052: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:22.052: INFO: hostexec-node1-fbn4g started at 2021-05-15 01:44:09 +0000 UTC (0+1 container statuses recorded) May 15 01:46:22.052: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:22.052: INFO: pod-828316bc-b350-4ddb-9ce9-cec4d8c6cd45 started at 2021-05-15 01:44:13 +0000 UTC (0+1 container statuses recorded) May 15 01:46:22.052: INFO: Container write-pod ready: true, restart count 0 May 15 01:46:22.052: INFO: pod-bfa4b2b7-b6e7-46d4-9222-2bd34e874439 started at 2021-05-15 01:44:21 +0000 UTC (0+1 container statuses recorded) May 15 01:46:22.052: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:22.052: INFO: pod-dc18f44a-4eff-4c02-9f90-6144d79f277b started at 2021-05-15 01:44:29 +0000 UTC (0+1 container statuses recorded) May 15 01:46:22.052: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:22.052: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:22.052: INFO: Init container install-cni ready: true, restart count 0 May 15 01:46:22.052: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:46:22.052: INFO: node-feature-discovery-worker-bw8zg started at 2021-05-15 00:18:56 +0000 UTC (0+1 container statuses recorded) May 15 01:46:22.052: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:46:22.052: INFO: collectd-mrzps started at 2021-05-15 00:19:22 +0000 UTC (0+3 container statuses recorded) May 15 01:46:22.052: INFO: Container collectd ready: true, restart count 0 May 15 01:46:22.052: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:46:22.052: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:46:22.052: INFO: hostexec-node1-lmdw4 started at 2021-05-15 01:46:14 +0000 UTC (0+1 container statuses recorded) May 15 01:46:22.052: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:22.052: INFO: pod-141a54ce-270d-46f4-a49c-b05dd96fbcdd started at 2021-05-15 01:44:28 +0000 UTC (0+1 container statuses recorded) May 15 01:46:22.052: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:22.052: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:22.052: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:22.052: INFO: prometheus-k8s-0 started at 2021-05-15 00:19:01 +0000 UTC (0+5 container statuses recorded) May 15 01:46:22.052: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 15 01:46:22.052: INFO: Container grafana ready: true, restart count 0 May 15 01:46:22.052: INFO: Container prometheus ready: true, restart count 26 May 15 01:46:22.052: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 15 01:46:22.052: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 15 01:46:22.052: INFO: pod-1812e695-95fc-4df1-8d69-0fbfa00893b8 started at 2021-05-15 01:44:26 +0000 UTC (0+1 container statuses recorded) May 15 01:46:22.052: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:22.052: INFO: hostexec-node1-zpl46 started at 2021-05-15 01:44:13 +0000 UTC (0+1 container statuses recorded) May 15 01:46:22.052: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:22.052: INFO: node-exporter-flvqz started at 2021-05-15 00:18:55 +0000 UTC (0+2 container statuses recorded) May 15 01:46:22.052: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:22.052: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:22.052: INFO: hostexec-node1-xrzq8 started at 2021-05-15 01:44:14 +0000 UTC (0+1 container statuses recorded) May 15 01:46:22.052: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:22.052: INFO: pod-c61f2051-9957-419e-995d-c3ed08d36192 started at 2021-05-15 01:46:20 +0000 UTC (0+1 container statuses recorded) May 15 01:46:22.052: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:22.052: INFO: cmk-4s6dm started at 2021-05-15 00:18:54 +0000 UTC (0+2 container statuses recorded) May 15 01:46:22.052: INFO: Container nodereport ready: true, restart count 0 May 15 01:46:22.052: INFO: Container reconcile ready: true, restart count 0 May 15 01:46:22.052: INFO: hostexec-node1-5mrx2 started at 2021-05-15 01:44:12 +0000 UTC (0+1 container statuses recorded) May 15 01:46:22.052: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:22.052: INFO: pod-1c1fa310-7f5b-4c20-8494-ee29c889ac5f started at 2021-05-15 01:44:17 +0000 UTC (0+1 container statuses recorded) May 15 01:46:22.052: INFO: Container write-pod ready: false, restart count 0 W0515 01:46:22.065682 32 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:22.115: INFO: Latency metrics for node node1 May 15 01:46:22.115: INFO: Logging node info for node node2 May 15 01:46:22.117: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 153990 0 2021-05-14 19:58:22 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-3052":"csi-mock-csi-mock-volumes-3052"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:46 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:44:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}},"f:status":{"f:volumesAttached":{}}}} {kubelet Update v1 2021-05-15 01:44:49 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}},"f:volumesInUse":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:19 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:19 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:19 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:19 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-3052^4],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-3052^4,DevicePath:,},},Config:nil,},} May 15 01:46:22.118: INFO: Logging kubelet events for node node2 May 15 01:46:22.123: INFO: Logging pods the kubelet thinks is on node node2 May 15 01:46:22.137: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:22.137: INFO: Init container install-cni ready: true, restart count 1 May 15 01:46:22.137: INFO: Container kube-flannel ready: true, restart count 4 May 15 01:46:22.137: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:46:22.137: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:22.137: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:22.137: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 15 01:46:22.137: INFO: Container tas-controller ready: true, restart count 0 May 15 01:46:22.137: INFO: Container tas-extender ready: true, restart count 0 May 15 01:46:22.137: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 15 01:46:22.137: INFO: Container collectd ready: true, restart count 0 May 15 01:46:22.137: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:46:22.137: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:46:22.137: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 15 01:46:22.137: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:46:22.137: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 15 01:46:22.137: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:46:22.137: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 15 01:46:22.137: INFO: Container nodereport ready: true, restart count 0 May 15 01:46:22.137: INFO: Container reconcile ready: true, restart count 0 May 15 01:46:22.137: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 15 01:46:22.137: INFO: Container cmk-webhook ready: true, restart count 0 May 15 01:46:22.137: INFO: csi-mockplugin-0 started at 2021-05-15 01:44:36 +0000 UTC (0+3 container statuses recorded) May 15 01:46:22.137: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:46:22.137: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:46:22.137: INFO: Container mock ready: true, restart count 0 May 15 01:46:22.137: INFO: csi-mockplugin-attacher-0 started at 2021-05-15 01:44:36 +0000 UTC (0+1 container statuses recorded) May 15 01:46:22.137: INFO: Container csi-attacher ready: true, restart count 0 May 15 01:46:22.137: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 15 01:46:22.137: INFO: Container discover ready: false, restart count 0 May 15 01:46:22.137: INFO: Container init ready: false, restart count 0 May 15 01:46:22.137: INFO: Container install ready: false, restart count 0 May 15 01:46:22.137: INFO: csi-mockplugin-0 started at 2021-05-15 01:46:19 +0000 UTC (0+3 container statuses recorded) May 15 01:46:22.137: INFO: Container csi-provisioner ready: false, restart count 0 May 15 01:46:22.137: INFO: Container driver-registrar ready: false, restart count 0 May 15 01:46:22.137: INFO: Container mock ready: false, restart count 0 May 15 01:46:22.137: INFO: csi-mockplugin-attacher-0 started at 2021-05-15 01:46:19 +0000 UTC (0+1 container statuses recorded) May 15 01:46:22.137: INFO: Container csi-attacher ready: false, restart count 0 May 15 01:46:22.137: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:22.137: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:46:22.137: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:22.137: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:22.137: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 15 01:46:22.137: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 15 01:46:22.137: INFO: pvc-volume-tester-8nftk started at 2021-05-15 01:44:47 +0000 UTC (0+1 container statuses recorded) May 15 01:46:22.137: INFO: Container volume-tester ready: true, restart count 0 May 15 01:46:22.137: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:46:22.137: INFO: Container nginx-proxy ready: true, restart count 2 W0515 01:46:22.149009 32 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:22.189: INFO: Latency metrics for node node2 May 15 01:46:22.189: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-6549" for this suite. • Failure [132.683 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: dir-link] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 Two pods mounting a local volume at the same time /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:244 should be able to write from pod1 and read from pod2 [It] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:245 May 15 01:46:21.718: Unexpected error: <*errors.errorString | 0xc000ab9180>: { s: "pod \"pod-bfa4b2b7-b6e7-46d4-9222-2bd34e874439\" is not Running: timed out waiting for the condition", } pod "pod-bfa4b2b7-b6e7-46d4-9222-2bd34e874439" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:727 ------------------------------ {"msg":"FAILED [sig-storage] PersistentVolumes-local [Volume type: dir-link] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2","total":-1,"completed":1,"skipped":41,"failed":1,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir-link] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2"]} SSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:43:58.347: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename csi-mock-volumes May 15 01:43:58.370: INFO: Found PodSecurityPolicies; testing pod creation to see if PodSecurityPolicy is enabled May 15 01:43:58.373: INFO: Error creating dryrun pod; assuming PodSecurityPolicy is disabled: admission webhook "cmk.intel.com" does not support dry run STEP: Waiting for a default service account to be provisioned in namespace [It] should expand volume without restarting pod if attach=on, nodeExpansion=on /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:552 STEP: Building a driver namespace object, basename csi-mock-volumes-7734 STEP: Waiting for a default service account to be provisioned in namespace STEP: deploying csi mock driver May 15 01:43:58.452: INFO: creating *v1.ServiceAccount: csi-mock-volumes-7734-857/csi-attacher May 15 01:43:58.454: INFO: creating *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-7734 May 15 01:43:58.454: INFO: Define cluster role external-attacher-runner-csi-mock-volumes-7734 May 15 01:43:58.457: INFO: creating *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-7734 May 15 01:43:58.460: INFO: creating *v1.Role: csi-mock-volumes-7734-857/external-attacher-cfg-csi-mock-volumes-7734 May 15 01:43:58.463: INFO: creating *v1.RoleBinding: csi-mock-volumes-7734-857/csi-attacher-role-cfg May 15 01:43:58.466: INFO: creating *v1.ServiceAccount: csi-mock-volumes-7734-857/csi-provisioner May 15 01:43:58.469: INFO: creating *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-7734 May 15 01:43:58.469: INFO: Define cluster role external-provisioner-runner-csi-mock-volumes-7734 May 15 01:43:58.472: INFO: creating *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-7734 May 15 01:43:58.474: INFO: creating *v1.Role: csi-mock-volumes-7734-857/external-provisioner-cfg-csi-mock-volumes-7734 May 15 01:43:58.476: INFO: creating *v1.RoleBinding: csi-mock-volumes-7734-857/csi-provisioner-role-cfg May 15 01:43:58.479: INFO: creating *v1.ServiceAccount: csi-mock-volumes-7734-857/csi-resizer May 15 01:43:58.482: INFO: creating *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-7734 May 15 01:43:58.482: INFO: Define cluster role external-resizer-runner-csi-mock-volumes-7734 May 15 01:43:58.485: INFO: creating *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-7734 May 15 01:43:58.487: INFO: creating *v1.Role: csi-mock-volumes-7734-857/external-resizer-cfg-csi-mock-volumes-7734 May 15 01:43:58.490: INFO: creating *v1.RoleBinding: csi-mock-volumes-7734-857/csi-resizer-role-cfg May 15 01:43:58.493: INFO: creating *v1.ServiceAccount: csi-mock-volumes-7734-857/csi-mock May 15 01:43:58.495: INFO: creating *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-7734 May 15 01:43:58.497: INFO: creating *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-7734 May 15 01:43:58.500: INFO: creating *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-7734 May 15 01:43:58.503: INFO: creating *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-7734 May 15 01:43:58.505: INFO: creating *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-7734 May 15 01:43:58.507: INFO: creating *v1.StorageClass: csi-mock-sc-csi-mock-volumes-7734 May 15 01:43:58.510: INFO: creating *v1.StatefulSet: csi-mock-volumes-7734-857/csi-mockplugin May 15 01:43:58.513: INFO: creating *v1.StatefulSet: csi-mock-volumes-7734-857/csi-mockplugin-attacher May 15 01:43:58.517: INFO: creating *v1.StatefulSet: csi-mock-volumes-7734-857/csi-mockplugin-resizer May 15 01:43:58.520: INFO: waiting for CSIDriver csi-mock-csi-mock-volumes-7734 to register on node node1 STEP: Creating pod May 15 01:44:24.919: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil May 15 01:44:24.925: INFO: Waiting up to 5m0s for PersistentVolumeClaims [pvc-7jbbd] to have phase Bound May 15 01:44:24.927: INFO: PersistentVolumeClaim pvc-7jbbd found but phase is Pending instead of Bound. May 15 01:44:26.930: INFO: PersistentVolumeClaim pvc-7jbbd found and phase=Bound (2.005111995s) STEP: Expanding current pvc STEP: Waiting for persistent volume resize to finish STEP: Waiting for PVC resize to finish STEP: Deleting pod pvc-volume-tester-t9kn9 May 15 01:45:54.973: INFO: Deleting pod "pvc-volume-tester-t9kn9" in namespace "csi-mock-volumes-7734" May 15 01:45:54.980: INFO: Wait up to 5m0s for pod "pvc-volume-tester-t9kn9" to be fully deleted STEP: Deleting claim pvc-7jbbd May 15 01:45:58.995: INFO: Waiting up to 2m0s for PersistentVolume pvc-f6d58bb6-53e5-44e0-8656-3be47b6c9bec to get deleted May 15 01:45:58.997: INFO: PersistentVolume pvc-f6d58bb6-53e5-44e0-8656-3be47b6c9bec found and phase=Bound (2.090399ms) May 15 01:46:01.000: INFO: PersistentVolume pvc-f6d58bb6-53e5-44e0-8656-3be47b6c9bec found and phase=Released (2.005811104s) May 15 01:46:03.004: INFO: PersistentVolume pvc-f6d58bb6-53e5-44e0-8656-3be47b6c9bec found and phase=Released (4.008903185s) May 15 01:46:05.006: INFO: PersistentVolume pvc-f6d58bb6-53e5-44e0-8656-3be47b6c9bec was removed STEP: Deleting storageclass csi-mock-volumes-7734-sc STEP: Cleaning up resources STEP: deleting the test namespace: csi-mock-volumes-7734 STEP: Waiting for namespaces [csi-mock-volumes-7734] to vanish STEP: uninstalling csi mock driver May 15 01:46:11.029: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-7734-857/csi-attacher May 15 01:46:11.035: INFO: deleting *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-7734 May 15 01:46:11.039: INFO: deleting *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-7734 May 15 01:46:11.044: INFO: deleting *v1.Role: csi-mock-volumes-7734-857/external-attacher-cfg-csi-mock-volumes-7734 May 15 01:46:11.049: INFO: deleting *v1.RoleBinding: csi-mock-volumes-7734-857/csi-attacher-role-cfg May 15 01:46:11.053: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-7734-857/csi-provisioner May 15 01:46:11.057: INFO: deleting *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-7734 May 15 01:46:11.061: INFO: deleting *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-7734 May 15 01:46:11.065: INFO: deleting *v1.Role: csi-mock-volumes-7734-857/external-provisioner-cfg-csi-mock-volumes-7734 May 15 01:46:11.070: INFO: deleting *v1.RoleBinding: csi-mock-volumes-7734-857/csi-provisioner-role-cfg May 15 01:46:11.074: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-7734-857/csi-resizer May 15 01:46:11.079: INFO: deleting *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-7734 May 15 01:46:11.083: INFO: deleting *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-7734 May 15 01:46:11.087: INFO: deleting *v1.Role: csi-mock-volumes-7734-857/external-resizer-cfg-csi-mock-volumes-7734 May 15 01:46:11.092: INFO: deleting *v1.RoleBinding: csi-mock-volumes-7734-857/csi-resizer-role-cfg May 15 01:46:11.096: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-7734-857/csi-mock May 15 01:46:11.100: INFO: deleting *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-7734 May 15 01:46:11.104: INFO: deleting *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-7734 May 15 01:46:11.108: INFO: deleting *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-7734 May 15 01:46:11.112: INFO: deleting *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-7734 May 15 01:46:11.116: INFO: deleting *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-7734 May 15 01:46:11.120: INFO: deleting *v1.StorageClass: csi-mock-sc-csi-mock-volumes-7734 May 15 01:46:11.124: INFO: deleting *v1.StatefulSet: csi-mock-volumes-7734-857/csi-mockplugin May 15 01:46:11.129: INFO: deleting *v1.StatefulSet: csi-mock-volumes-7734-857/csi-mockplugin-attacher May 15 01:46:11.133: INFO: deleting *v1.StatefulSet: csi-mock-volumes-7734-857/csi-mockplugin-resizer STEP: deleting the driver namespace: csi-mock-volumes-7734-857 STEP: Waiting for namespaces [csi-mock-volumes-7734-857] to vanish [AfterEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:46:23.145: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "csi-mock-volumes-7734" for this suite. May 15 01:46:23.151: INFO: Namespace csi-mock-volumes-7734 was already deleted STEP: Destroying namespace "csi-mock-volumes-7734-857" for this suite. May 15 01:46:23.153: INFO: Namespace csi-mock-volumes-7734-857 was already deleted • [SLOW TEST:144.807 seconds] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 CSI online volume expansion /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:537 should expand volume without restarting pod if attach=on, nodeExpansion=on /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:552 ------------------------------ {"msg":"PASSED [sig-storage] CSI mock volume CSI online volume expansion should expand volume without restarting pod if attach=on, nodeExpansion=on","total":-1,"completed":1,"skipped":2,"failed":0} SSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:44:12.914: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: dir] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes May 15 01:44:22.960: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-aa348624-9e1c-44b8-9072-ac2c69acc07c] Namespace:persistent-local-volumes-test-3319 PodName:hostexec-node1-5mrx2 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:44:22.961: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:44:23.072: INFO: Creating a PV followed by a PVC May 15 01:44:23.080: INFO: Waiting for PV local-pvfmj2b to bind to PVC pvc-5hzbn May 15 01:44:23.080: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-5hzbn] to have phase Bound May 15 01:44:23.082: INFO: PersistentVolumeClaim pvc-5hzbn found but phase is Pending instead of Bound. May 15 01:44:25.085: INFO: PersistentVolumeClaim pvc-5hzbn found and phase=Bound (2.004950306s) May 15 01:44:25.085: INFO: Waiting up to 3m0s for PersistentVolume local-pvfmj2b to have phase Bound May 15 01:44:25.087: INFO: PersistentVolume local-pvfmj2b found and phase=Bound (2.078993ms) [BeforeEach] One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:211 STEP: Creating pod1 STEP: Creating a pod May 15 01:46:25.110: FAIL: Unexpected error: <*errors.errorString | 0xc0045d29b0>: { s: "pod \"pod-38d6e6d5-eb01-4ca7-89a4-5da7e94c8500\" is not Running: timed out waiting for the condition", } pod "pod-38d6e6d5-eb01-4ca7-89a4-5da7e94c8500" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage.glob..func20.2.3.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 +0x119 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc001881380) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc001881380) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc001881380, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:223 STEP: Deleting pod1 STEP: Deleting pod pod-38d6e6d5-eb01-4ca7-89a4-5da7e94c8500 in namespace persistent-local-volumes-test-3319 [AfterEach] [Volume type: dir] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:46:25.119: INFO: Deleting PersistentVolumeClaim "pvc-5hzbn" May 15 01:46:25.123: INFO: Deleting PersistentVolume "local-pvfmj2b" STEP: Removing the test directory May 15 01:46:25.126: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-aa348624-9e1c-44b8-9072-ac2c69acc07c] Namespace:persistent-local-volumes-test-3319 PodName:hostexec-node1-5mrx2 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:46:25.126: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "persistent-local-volumes-test-3319". STEP: Found 12 events. May 15 01:46:26.092: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for hostexec-node1-5mrx2: { } Scheduled: Successfully assigned persistent-local-volumes-test-3319/hostexec-node1-5mrx2 to node1 May 15 01:46:26.092: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-38d6e6d5-eb01-4ca7-89a4-5da7e94c8500: { } Scheduled: Successfully assigned persistent-local-volumes-test-3319/pod-38d6e6d5-eb01-4ca7-89a4-5da7e94c8500 to node1 May 15 01:46:26.092: INFO: At 2021-05-15 01:44:13 +0000 UTC - event for hostexec-node1-5mrx2: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 15 01:46:26.092: INFO: At 2021-05-15 01:44:15 +0000 UTC - event for hostexec-node1-5mrx2: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 2.104046734s May 15 01:46:26.092: INFO: At 2021-05-15 01:44:15 +0000 UTC - event for hostexec-node1-5mrx2: {kubelet node1} Created: Created container agnhost-container May 15 01:46:26.092: INFO: At 2021-05-15 01:44:16 +0000 UTC - event for hostexec-node1-5mrx2: {kubelet node1} Started: Started container agnhost-container May 15 01:46:26.092: INFO: At 2021-05-15 01:44:28 +0000 UTC - event for pod-38d6e6d5-eb01-4ca7-89a4-5da7e94c8500: {multus } AddedInterface: Add eth0 [10.244.3.125/24] May 15 01:46:26.092: INFO: At 2021-05-15 01:44:28 +0000 UTC - event for pod-38d6e6d5-eb01-4ca7-89a4-5da7e94c8500: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:46:26.092: INFO: At 2021-05-15 01:44:29 +0000 UTC - event for pod-38d6e6d5-eb01-4ca7-89a4-5da7e94c8500: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:46:26.092: INFO: At 2021-05-15 01:44:29 +0000 UTC - event for pod-38d6e6d5-eb01-4ca7-89a4-5da7e94c8500: {kubelet node1} Failed: Error: ErrImagePull May 15 01:46:26.092: INFO: At 2021-05-15 01:44:29 +0000 UTC - event for pod-38d6e6d5-eb01-4ca7-89a4-5da7e94c8500: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 15 01:46:26.092: INFO: At 2021-05-15 01:44:29 +0000 UTC - event for pod-38d6e6d5-eb01-4ca7-89a4-5da7e94c8500: {kubelet node1} Failed: Error: ImagePullBackOff May 15 01:46:26.094: INFO: POD NODE PHASE GRACE CONDITIONS May 15 01:46:26.094: INFO: hostexec-node1-5mrx2 node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:12 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:17 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:17 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:12 +0000 UTC }] May 15 01:46:26.094: INFO: pod-38d6e6d5-eb01-4ca7-89a4-5da7e94c8500 node1 Pending 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:25 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:25 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:25 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:25 +0000 UTC }] May 15 01:46:26.094: INFO: May 15 01:46:26.099: INFO: Logging node info for node master1 May 15 01:46:26.101: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 153911 0 2021-05-14 19:56:35 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:17 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:17 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:17 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:17 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[@ :],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:46:26.102: INFO: Logging kubelet events for node master1 May 15 01:46:26.106: INFO: Logging pods the kubelet thinks is on node master1 May 15 01:46:26.127: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 15 01:46:26.127: INFO: Container docker-registry ready: true, restart count 0 May 15 01:46:26.127: INFO: Container nginx ready: true, restart count 0 May 15 01:46:26.127: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.127: INFO: Container nfd-controller ready: true, restart count 0 May 15 01:46:26.127: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.127: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:46:26.127: INFO: kubernetes-metrics-scraper-678c97765c-fswrn started at 2021-05-15 00:18:49 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.127: INFO: Container kubernetes-metrics-scraper ready: true, restart count 0 May 15 01:46:26.127: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.127: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:46:26.127: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.127: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:26.127: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.127: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:46:26.127: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:26.127: INFO: Init container install-cni ready: true, restart count 0 May 15 01:46:26.127: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:46:26.127: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:46:26.127: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:26.127: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:26.127: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.127: INFO: Container kube-scheduler ready: true, restart count 0 W0515 01:46:26.139349 24 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:26.166: INFO: Latency metrics for node master1 May 15 01:46:26.166: INFO: Logging node info for node master2 May 15 01:46:26.169: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 154162 0 2021-05-14 19:57:04 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:25 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:25 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:25 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:25 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:46:26.169: INFO: Logging kubelet events for node master2 May 15 01:46:26.174: INFO: Logging pods the kubelet thinks is on node master2 May 15 01:46:26.182: INFO: coredns-7677f9bb54-96w24 started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.182: INFO: Container coredns ready: true, restart count 0 May 15 01:46:26.182: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.182: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:46:26.182: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.182: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:26.182: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.182: INFO: Container kube-scheduler ready: true, restart count 2 May 15 01:46:26.182: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:26.182: INFO: Init container install-cni ready: true, restart count 0 May 15 01:46:26.182: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:46:26.182: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.182: INFO: Container autoscaler ready: true, restart count 2 May 15 01:46:26.182: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:46:26.182: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:26.182: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:26.183: INFO: prometheus-operator-5bb8cb9d8f-fqb87 started at 2021-05-15 00:18:49 +0000 UTC (0+2 container statuses recorded) May 15 01:46:26.183: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:26.183: INFO: Container prometheus-operator ready: true, restart count 0 May 15 01:46:26.183: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.183: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:46:26.183: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.183: INFO: Container kube-controller-manager ready: true, restart count 2 W0515 01:46:26.195313 24 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:26.227: INFO: Latency metrics for node master2 May 15 01:46:26.227: INFO: Logging node info for node master3 May 15 01:46:26.230: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 154124 0 2021-05-14 19:57:14 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234767872 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324603904 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:23 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:23 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:23 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:23 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:46:26.230: INFO: Logging kubelet events for node master3 May 15 01:46:26.234: INFO: Logging pods the kubelet thinks is on node master3 May 15 01:46:26.242: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.242: INFO: Container kube-proxy ready: true, restart count 1 May 15 01:46:26.242: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:26.242: INFO: Init container install-cni ready: true, restart count 0 May 15 01:46:26.242: INFO: Container kube-flannel ready: true, restart count 2 May 15 01:46:26.242: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.242: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:26.242: INFO: coredns-7677f9bb54-rpj8c started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.242: INFO: Container coredns ready: true, restart count 0 May 15 01:46:26.242: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:46:26.242: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:26.242: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:26.242: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.242: INFO: Container kube-controller-manager ready: true, restart count 3 May 15 01:46:26.242: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.242: INFO: Container kube-scheduler ready: true, restart count 3 May 15 01:46:26.242: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.242: INFO: Container kube-apiserver ready: true, restart count 0 W0515 01:46:26.254728 24 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:26.286: INFO: Latency metrics for node master3 May 15 01:46:26.287: INFO: Logging node info for node node1 May 15 01:46:26.289: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 154091 0 2021-05-14 19:58:21 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-7734":"csi-mock-csi-mock-volumes-7734"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:00 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:44:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:46:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:23 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:23 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:23 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:23 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[@ :],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:46:26.290: INFO: Logging kubelet events for node node1 May 15 01:46:26.296: INFO: Logging pods the kubelet thinks is on node node1 May 15 01:46:26.313: INFO: pod-141a54ce-270d-46f4-a49c-b05dd96fbcdd started at 2021-05-15 01:44:28 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.313: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:26.313: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:26.313: INFO: Init container install-cni ready: true, restart count 0 May 15 01:46:26.313: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:46:26.313: INFO: node-feature-discovery-worker-bw8zg started at 2021-05-15 00:18:56 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.313: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:46:26.313: INFO: collectd-mrzps started at 2021-05-15 00:19:22 +0000 UTC (0+3 container statuses recorded) May 15 01:46:26.313: INFO: Container collectd ready: true, restart count 0 May 15 01:46:26.313: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:46:26.313: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:46:26.313: INFO: hostexec-node1-lmdw4 started at 2021-05-15 01:46:14 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.313: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:26.313: INFO: hostexec-node1-zpl46 started at 2021-05-15 01:44:13 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.313: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:26.313: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.313: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:26.313: INFO: prometheus-k8s-0 started at 2021-05-15 00:19:01 +0000 UTC (0+5 container statuses recorded) May 15 01:46:26.313: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 15 01:46:26.313: INFO: Container grafana ready: true, restart count 0 May 15 01:46:26.313: INFO: Container prometheus ready: true, restart count 26 May 15 01:46:26.313: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 15 01:46:26.313: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 15 01:46:26.313: INFO: pod-1812e695-95fc-4df1-8d69-0fbfa00893b8 started at 2021-05-15 01:44:26 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.313: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:26.313: INFO: hostexec-node1-5fw6h started at 2021-05-15 01:46:22 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.313: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:26.313: INFO: node-exporter-flvqz started at 2021-05-15 00:18:55 +0000 UTC (0+2 container statuses recorded) May 15 01:46:26.313: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:26.313: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:26.313: INFO: hostexec-node1-xrzq8 started at 2021-05-15 01:44:14 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.313: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:26.313: INFO: pod-c61f2051-9957-419e-995d-c3ed08d36192 started at 2021-05-15 01:46:20 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.313: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:26.313: INFO: cmk-4s6dm started at 2021-05-15 00:18:54 +0000 UTC (0+2 container statuses recorded) May 15 01:46:26.313: INFO: Container nodereport ready: true, restart count 0 May 15 01:46:26.313: INFO: Container reconcile ready: true, restart count 0 May 15 01:46:26.313: INFO: hostexec-node1-5mrx2 started at 2021-05-15 01:44:12 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.313: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:26.313: INFO: hostexec-node1-dfp4d started at 2021-05-15 01:44:32 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.313: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:26.313: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.313: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:46:26.313: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4vxcc started at 2021-05-15 00:19:00 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.313: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:46:26.313: INFO: hostexec-node1-fbn4g started at 2021-05-15 01:44:09 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.313: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:26.313: INFO: pod-828316bc-b350-4ddb-9ce9-cec4d8c6cd45 started at 2021-05-15 01:44:13 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.313: INFO: Container write-pod ready: true, restart count 0 May 15 01:46:26.313: INFO: pod-38d6e6d5-eb01-4ca7-89a4-5da7e94c8500 started at 2021-05-15 01:44:25 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.313: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:26.313: INFO: pod-e1744dfa-1547-4ca5-acdf-8eb7113868b1 started at 2021-05-15 01:44:42 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.313: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:26.313: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.313: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:46:26.313: INFO: hostexec-node1-dzx4r started at 2021-05-15 01:43:58 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.313: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:26.313: INFO: pod-bfa4b2b7-b6e7-46d4-9222-2bd34e874439 started at 2021-05-15 01:44:21 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.313: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:26.313: INFO: pod-dc18f44a-4eff-4c02-9f90-6144d79f277b started at 2021-05-15 01:44:29 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.313: INFO: Container write-pod ready: false, restart count 0 W0515 01:46:26.326399 24 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:26.363: INFO: Latency metrics for node node1 May 15 01:46:26.363: INFO: Logging node info for node node2 May 15 01:46:26.366: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 154137 0 2021-05-14 19:58:22 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-3052":"csi-mock-csi-mock-volumes-3052","csi-mock-csi-mock-volumes-8760":"csi-mock-csi-mock-volumes-8760"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:46 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:44:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}},"f:status":{"f:volumesAttached":{}}}} {kubelet Update v1 2021-05-15 01:44:49 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}},"f:volumesInUse":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:19 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:19 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:19 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:19 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-3052^4],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-3052^4,DevicePath:,},},Config:nil,},} May 15 01:46:26.366: INFO: Logging kubelet events for node node2 May 15 01:46:26.370: INFO: Logging pods the kubelet thinks is on node node2 May 15 01:46:26.389: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 15 01:46:26.389: INFO: Container tas-controller ready: true, restart count 0 May 15 01:46:26.389: INFO: Container tas-extender ready: true, restart count 0 May 15 01:46:26.389: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 15 01:46:26.389: INFO: Container collectd ready: true, restart count 0 May 15 01:46:26.389: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:46:26.389: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:46:26.389: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.389: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:46:26.389: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.389: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:46:26.390: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 15 01:46:26.390: INFO: Container nodereport ready: true, restart count 0 May 15 01:46:26.390: INFO: Container reconcile ready: true, restart count 0 May 15 01:46:26.390: INFO: pod-05c687ae-4432-4531-8186-a9b33485c6d8 started at 2021-05-15 01:46:23 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.390: INFO: Container test-container ready: false, restart count 0 May 15 01:46:26.390: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.390: INFO: Container cmk-webhook ready: true, restart count 0 May 15 01:46:26.390: INFO: pvc-volume-tester-jq8ps started at 2021-05-15 01:46:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.390: INFO: Container volume-tester ready: false, restart count 0 May 15 01:46:26.390: INFO: csi-mockplugin-0 started at 2021-05-15 01:44:36 +0000 UTC (0+3 container statuses recorded) May 15 01:46:26.390: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:46:26.390: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:46:26.390: INFO: Container mock ready: true, restart count 0 May 15 01:46:26.390: INFO: csi-mockplugin-attacher-0 started at 2021-05-15 01:44:36 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.390: INFO: Container csi-attacher ready: true, restart count 0 May 15 01:46:26.390: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 15 01:46:26.390: INFO: Container discover ready: false, restart count 0 May 15 01:46:26.390: INFO: Container init ready: false, restart count 0 May 15 01:46:26.390: INFO: Container install ready: false, restart count 0 May 15 01:46:26.390: INFO: csi-mockplugin-0 started at 2021-05-15 01:46:19 +0000 UTC (0+3 container statuses recorded) May 15 01:46:26.390: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:46:26.390: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:46:26.390: INFO: Container mock ready: true, restart count 0 May 15 01:46:26.390: INFO: csi-mockplugin-attacher-0 started at 2021-05-15 01:46:19 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.390: INFO: Container csi-attacher ready: true, restart count 0 May 15 01:46:26.390: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.390: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:46:26.390: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.390: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:26.390: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.390: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 15 01:46:26.390: INFO: pvc-volume-tester-8nftk started at 2021-05-15 01:44:47 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.390: INFO: Container volume-tester ready: true, restart count 0 May 15 01:46:26.390: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.390: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:46:26.390: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:26.390: INFO: Init container install-cni ready: true, restart count 1 May 15 01:46:26.390: INFO: Container kube-flannel ready: true, restart count 4 May 15 01:46:26.390: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:46:26.390: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:26.390: INFO: Container node-exporter ready: true, restart count 0 W0515 01:46:26.403375 24 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:26.458: INFO: Latency metrics for node node2 May 15 01:46:26.458: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-3319" for this suite. • Failure in Spec Setup (BeforeEach) [133.552 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: dir] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:205 should be able to mount volume and write from pod1 [BeforeEach] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:234 May 15 01:46:25.110: Unexpected error: <*errors.errorString | 0xc0045d29b0>: { s: "pod \"pod-38d6e6d5-eb01-4ca7-89a4-5da7e94c8500\" is not Running: timed out waiting for the condition", } pod "pod-38d6e6d5-eb01-4ca7-89a4-5da7e94c8500" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 ------------------------------ {"msg":"FAILED [sig-storage] PersistentVolumes-local [Volume type: dir] One pod requesting one prebound PVC should be able to mount volume and write from pod1","total":-1,"completed":2,"skipped":507,"failed":1,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir] One pod requesting one prebound PVC should be able to mount volume and write from pod1"]} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:43:58.353: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test May 15 01:43:58.374: INFO: Found PodSecurityPolicies; testing pod creation to see if PodSecurityPolicy is enabled May 15 01:43:58.376: INFO: Error creating dryrun pod; assuming PodSecurityPolicy is disabled: admission webhook "cmk.intel.com" does not support dry run STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: dir-bindmounted] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes May 15 01:44:02.415: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir /tmp/local-volume-test-e96b9c7a-2aac-4ceb-8954-60298d2929d9 && mount --bind /tmp/local-volume-test-e96b9c7a-2aac-4ceb-8954-60298d2929d9 /tmp/local-volume-test-e96b9c7a-2aac-4ceb-8954-60298d2929d9] Namespace:persistent-local-volumes-test-1942 PodName:hostexec-node1-dzx4r ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:44:02.415: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:44:03.050: INFO: Creating a PV followed by a PVC May 15 01:44:03.058: INFO: Waiting for PV local-pvmmqcz to bind to PVC pvc-pm22z May 15 01:44:03.059: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-pm22z] to have phase Bound May 15 01:44:03.061: INFO: PersistentVolumeClaim pvc-pm22z found but phase is Pending instead of Bound. May 15 01:44:05.065: INFO: PersistentVolumeClaim pvc-pm22z found but phase is Pending instead of Bound. May 15 01:44:07.068: INFO: PersistentVolumeClaim pvc-pm22z found but phase is Pending instead of Bound. May 15 01:44:09.072: INFO: PersistentVolumeClaim pvc-pm22z found but phase is Pending instead of Bound. May 15 01:44:11.076: INFO: PersistentVolumeClaim pvc-pm22z found but phase is Pending instead of Bound. May 15 01:44:13.081: INFO: PersistentVolumeClaim pvc-pm22z found and phase=Bound (10.02286027s) May 15 01:44:13.081: INFO: Waiting up to 3m0s for PersistentVolume local-pvmmqcz to have phase Bound May 15 01:44:13.086: INFO: PersistentVolume local-pvmmqcz found and phase=Bound (4.704961ms) [It] should be able to write from pod1 and read from pod2 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:245 STEP: Creating pod1 to write to the PV STEP: Creating a pod May 15 01:44:25.116: INFO: pod "pod-828316bc-b350-4ddb-9ce9-cec4d8c6cd45" created on Node "node1" STEP: Writing in pod1 May 15 01:44:25.116: INFO: ExecWithOptions {Command:[/bin/sh -c mkdir -p /mnt/volume1; echo test-file-content > /mnt/volume1/test-file] Namespace:persistent-local-volumes-test-1942 PodName:pod-828316bc-b350-4ddb-9ce9-cec4d8c6cd45 ContainerName:write-pod Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:false} May 15 01:44:25.116: INFO: >>> kubeConfig: /root/.kube/config May 15 01:44:26.156: INFO: podRWCmdExec cmd: "mkdir -p /mnt/volume1; echo test-file-content > /mnt/volume1/test-file", out: "", stderr: "", err: May 15 01:44:26.156: INFO: ExecWithOptions {Command:[/bin/sh -c cat /mnt/volume1/test-file] Namespace:persistent-local-volumes-test-1942 PodName:pod-828316bc-b350-4ddb-9ce9-cec4d8c6cd45 ContainerName:write-pod Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:false} May 15 01:44:26.156: INFO: >>> kubeConfig: /root/.kube/config May 15 01:44:26.292: INFO: podRWCmdExec cmd: "cat /mnt/volume1/test-file", out: "test-file-content", stderr: "", err: STEP: Creating pod2 to read from the PV STEP: Creating a pod May 15 01:46:26.314: FAIL: Unexpected error: <*errors.errorString | 0xc00436c570>: { s: "pod \"pod-1812e695-95fc-4df1-8d69-0fbfa00893b8\" is not Running: timed out waiting for the condition", } pod "pod-1812e695-95fc-4df1-8d69-0fbfa00893b8" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage.twoPodsReadWriteTest(0xc001b72840, 0xc00479e280, 0xc0044fa930) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:740 +0x2b9 k8s.io/kubernetes/test/e2e/storage.glob..func20.2.4.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:246 +0x45 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc002904000) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc002904000) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc002904000, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [Volume type: dir-bindmounted] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:46:26.316: INFO: Deleting PersistentVolumeClaim "pvc-pm22z" May 15 01:46:26.319: INFO: Deleting PersistentVolume "local-pvmmqcz" STEP: Removing the test directory May 15 01:46:26.323: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c umount /tmp/local-volume-test-e96b9c7a-2aac-4ceb-8954-60298d2929d9 && rm -r /tmp/local-volume-test-e96b9c7a-2aac-4ceb-8954-60298d2929d9] Namespace:persistent-local-volumes-test-1942 PodName:hostexec-node1-dzx4r ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:46:26.323: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "persistent-local-volumes-test-1942". STEP: Found 23 events. May 15 01:46:26.454: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for hostexec-node1-dzx4r: { } Scheduled: Successfully assigned persistent-local-volumes-test-1942/hostexec-node1-dzx4r to node1 May 15 01:46:26.454: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-1812e695-95fc-4df1-8d69-0fbfa00893b8: { } Scheduled: Successfully assigned persistent-local-volumes-test-1942/pod-1812e695-95fc-4df1-8d69-0fbfa00893b8 to node1 May 15 01:46:26.454: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-828316bc-b350-4ddb-9ce9-cec4d8c6cd45: { } Scheduled: Successfully assigned persistent-local-volumes-test-1942/pod-828316bc-b350-4ddb-9ce9-cec4d8c6cd45 to node1 May 15 01:46:26.454: INFO: At 2021-05-15 01:43:59 +0000 UTC - event for hostexec-node1-dzx4r: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 15 01:46:26.454: INFO: At 2021-05-15 01:43:59 +0000 UTC - event for hostexec-node1-dzx4r: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 630.415947ms May 15 01:46:26.454: INFO: At 2021-05-15 01:44:00 +0000 UTC - event for hostexec-node1-dzx4r: {kubelet node1} Created: Created container agnhost-container May 15 01:46:26.454: INFO: At 2021-05-15 01:44:00 +0000 UTC - event for hostexec-node1-dzx4r: {kubelet node1} Started: Started container agnhost-container May 15 01:46:26.454: INFO: At 2021-05-15 01:44:03 +0000 UTC - event for pvc-pm22z: {persistentvolume-controller } ProvisioningFailed: no volume plugin matched name: kubernetes.io/no-provisioner May 15 01:46:26.454: INFO: At 2021-05-15 01:44:17 +0000 UTC - event for pod-828316bc-b350-4ddb-9ce9-cec4d8c6cd45: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:46:26.454: INFO: At 2021-05-15 01:44:17 +0000 UTC - event for pod-828316bc-b350-4ddb-9ce9-cec4d8c6cd45: {multus } AddedInterface: Add eth0 [10.244.3.121/24] May 15 01:46:26.454: INFO: At 2021-05-15 01:44:18 +0000 UTC - event for pod-828316bc-b350-4ddb-9ce9-cec4d8c6cd45: {kubelet node1} Pulled: Successfully pulled image "docker.io/library/busybox:1.29" in 1.340957734s May 15 01:46:26.454: INFO: At 2021-05-15 01:44:19 +0000 UTC - event for pod-828316bc-b350-4ddb-9ce9-cec4d8c6cd45: {kubelet node1} Created: Created container write-pod May 15 01:46:26.454: INFO: At 2021-05-15 01:44:19 +0000 UTC - event for pod-828316bc-b350-4ddb-9ce9-cec4d8c6cd45: {kubelet node1} Started: Started container write-pod May 15 01:46:26.454: INFO: At 2021-05-15 01:44:29 +0000 UTC - event for pod-1812e695-95fc-4df1-8d69-0fbfa00893b8: {multus } AddedInterface: Add eth0 [10.244.3.126/24] May 15 01:46:26.454: INFO: At 2021-05-15 01:44:29 +0000 UTC - event for pod-1812e695-95fc-4df1-8d69-0fbfa00893b8: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:46:26.454: INFO: At 2021-05-15 01:44:30 +0000 UTC - event for pod-1812e695-95fc-4df1-8d69-0fbfa00893b8: {kubelet node1} Failed: Error: ErrImagePull May 15 01:46:26.455: INFO: At 2021-05-15 01:44:30 +0000 UTC - event for pod-1812e695-95fc-4df1-8d69-0fbfa00893b8: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:46:26.455: INFO: At 2021-05-15 01:44:31 +0000 UTC - event for pod-1812e695-95fc-4df1-8d69-0fbfa00893b8: {kubelet node1} SandboxChanged: Pod sandbox changed, it will be killed and re-created. May 15 01:46:26.455: INFO: At 2021-05-15 01:44:34 +0000 UTC - event for pod-1812e695-95fc-4df1-8d69-0fbfa00893b8: {multus } AddedInterface: Add eth0 [10.244.3.128/24] May 15 01:46:26.455: INFO: At 2021-05-15 01:44:34 +0000 UTC - event for pod-1812e695-95fc-4df1-8d69-0fbfa00893b8: {kubelet node1} Failed: Error: ImagePullBackOff May 15 01:46:26.455: INFO: At 2021-05-15 01:44:34 +0000 UTC - event for pod-1812e695-95fc-4df1-8d69-0fbfa00893b8: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 15 01:46:26.455: INFO: At 2021-05-15 01:44:37 +0000 UTC - event for pod-1812e695-95fc-4df1-8d69-0fbfa00893b8: {multus } AddedInterface: Add eth0 [10.244.3.131/24] May 15 01:46:26.455: INFO: At 2021-05-15 01:44:42 +0000 UTC - event for pod-1812e695-95fc-4df1-8d69-0fbfa00893b8: {multus } AddedInterface: Add eth0 [10.244.3.135/24] May 15 01:46:26.458: INFO: POD NODE PHASE GRACE CONDITIONS May 15 01:46:26.458: INFO: hostexec-node1-dzx4r node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:43:58 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:01 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:01 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:43:58 +0000 UTC }] May 15 01:46:26.458: INFO: pod-1812e695-95fc-4df1-8d69-0fbfa00893b8 node1 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:26 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:26 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:26 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:26 +0000 UTC }] May 15 01:46:26.458: INFO: pod-828316bc-b350-4ddb-9ce9-cec4d8c6cd45 node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:20 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:20 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:13 +0000 UTC }] May 15 01:46:26.458: INFO: May 15 01:46:26.462: INFO: Logging node info for node master1 May 15 01:46:26.465: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 153911 0 2021-05-14 19:56:35 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:17 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:17 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:17 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:17 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[@ :],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:46:26.465: INFO: Logging kubelet events for node master1 May 15 01:46:26.470: INFO: Logging pods the kubelet thinks is on node master1 May 15 01:46:26.492: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.492: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:46:26.492: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.492: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:26.492: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 15 01:46:26.492: INFO: Container docker-registry ready: true, restart count 0 May 15 01:46:26.492: INFO: Container nginx ready: true, restart count 0 May 15 01:46:26.492: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.492: INFO: Container nfd-controller ready: true, restart count 0 May 15 01:46:26.492: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.492: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:46:26.493: INFO: kubernetes-metrics-scraper-678c97765c-fswrn started at 2021-05-15 00:18:49 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.493: INFO: Container kubernetes-metrics-scraper ready: true, restart count 0 May 15 01:46:26.493: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:46:26.493: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:26.493: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:26.493: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.493: INFO: Container kube-scheduler ready: true, restart count 0 May 15 01:46:26.493: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.493: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:46:26.493: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:26.493: INFO: Init container install-cni ready: true, restart count 0 May 15 01:46:26.493: INFO: Container kube-flannel ready: true, restart count 1 W0515 01:46:26.503139 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:26.531: INFO: Latency metrics for node master1 May 15 01:46:26.531: INFO: Logging node info for node master2 May 15 01:46:26.533: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 154162 0 2021-05-14 19:57:04 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:25 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:25 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:25 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:25 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:46:26.534: INFO: Logging kubelet events for node master2 May 15 01:46:26.538: INFO: Logging pods the kubelet thinks is on node master2 May 15 01:46:26.548: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.548: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:46:26.548: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.548: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:26.548: INFO: coredns-7677f9bb54-96w24 started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.548: INFO: Container coredns ready: true, restart count 0 May 15 01:46:26.548: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:46:26.548: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:26.548: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:26.548: INFO: prometheus-operator-5bb8cb9d8f-fqb87 started at 2021-05-15 00:18:49 +0000 UTC (0+2 container statuses recorded) May 15 01:46:26.548: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:26.548: INFO: Container prometheus-operator ready: true, restart count 0 May 15 01:46:26.548: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.548: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:46:26.548: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.548: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:46:26.548: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.548: INFO: Container kube-scheduler ready: true, restart count 2 May 15 01:46:26.548: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:26.548: INFO: Init container install-cni ready: true, restart count 0 May 15 01:46:26.548: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:46:26.548: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.548: INFO: Container autoscaler ready: true, restart count 2 W0515 01:46:26.560599 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:26.583: INFO: Latency metrics for node master2 May 15 01:46:26.583: INFO: Logging node info for node master3 May 15 01:46:26.586: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 154124 0 2021-05-14 19:57:14 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234767872 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324603904 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:23 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:23 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:23 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:23 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:46:26.586: INFO: Logging kubelet events for node master3 May 15 01:46:26.589: INFO: Logging pods the kubelet thinks is on node master3 May 15 01:46:26.598: INFO: coredns-7677f9bb54-rpj8c started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.598: INFO: Container coredns ready: true, restart count 0 May 15 01:46:26.598: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:46:26.598: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:26.598: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:26.598: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.598: INFO: Container kube-controller-manager ready: true, restart count 3 May 15 01:46:26.598: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.598: INFO: Container kube-scheduler ready: true, restart count 3 May 15 01:46:26.598: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.598: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:46:26.598: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.598: INFO: Container kube-proxy ready: true, restart count 1 May 15 01:46:26.598: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:26.598: INFO: Init container install-cni ready: true, restart count 0 May 15 01:46:26.598: INFO: Container kube-flannel ready: true, restart count 2 May 15 01:46:26.598: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.598: INFO: Container kube-multus ready: true, restart count 1 W0515 01:46:26.612036 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:26.640: INFO: Latency metrics for node master3 May 15 01:46:26.640: INFO: Logging node info for node node1 May 15 01:46:26.644: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 154091 0 2021-05-14 19:58:21 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-7734":"csi-mock-csi-mock-volumes-7734"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:00 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:44:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:46:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:23 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:23 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:23 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:23 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[@ :],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:46:26.644: INFO: Logging kubelet events for node node1 May 15 01:46:26.649: INFO: Logging pods the kubelet thinks is on node node1 May 15 01:46:26.666: INFO: prometheus-k8s-0 started at 2021-05-15 00:19:01 +0000 UTC (0+5 container statuses recorded) May 15 01:46:26.666: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 15 01:46:26.666: INFO: Container grafana ready: true, restart count 0 May 15 01:46:26.666: INFO: Container prometheus ready: true, restart count 26 May 15 01:46:26.666: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 15 01:46:26.666: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 15 01:46:26.666: INFO: pod-1812e695-95fc-4df1-8d69-0fbfa00893b8 started at 2021-05-15 01:44:26 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.666: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:26.666: INFO: hostexec-node1-5fw6h started at 2021-05-15 01:46:22 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.666: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:26.666: INFO: hostexec-node1-zpl46 started at 2021-05-15 01:44:13 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.666: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:26.666: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.666: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:26.666: INFO: hostexec-node1-xrzq8 started at 2021-05-15 01:44:14 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.666: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:26.666: INFO: pod-c61f2051-9957-419e-995d-c3ed08d36192 started at 2021-05-15 01:46:20 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.666: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:26.666: INFO: node-exporter-flvqz started at 2021-05-15 00:18:55 +0000 UTC (0+2 container statuses recorded) May 15 01:46:26.666: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:26.666: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:26.666: INFO: cmk-4s6dm started at 2021-05-15 00:18:54 +0000 UTC (0+2 container statuses recorded) May 15 01:46:26.666: INFO: Container nodereport ready: true, restart count 0 May 15 01:46:26.666: INFO: Container reconcile ready: true, restart count 0 May 15 01:46:26.666: INFO: hostexec-node1-5mrx2 started at 2021-05-15 01:44:12 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.666: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:26.666: INFO: hostexec-node1-dfp4d started at 2021-05-15 01:44:32 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.666: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:26.666: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4vxcc started at 2021-05-15 00:19:00 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.666: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:46:26.666: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.666: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:46:26.666: INFO: hostexec-node1-dzx4r started at 2021-05-15 01:43:58 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.667: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:26.667: INFO: hostexec-node1-fbn4g started at 2021-05-15 01:44:09 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.667: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:26.667: INFO: pod-828316bc-b350-4ddb-9ce9-cec4d8c6cd45 started at 2021-05-15 01:44:13 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.667: INFO: Container write-pod ready: true, restart count 0 May 15 01:46:26.667: INFO: pod-38d6e6d5-eb01-4ca7-89a4-5da7e94c8500 started at 2021-05-15 01:44:25 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.667: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:26.667: INFO: pod-e1744dfa-1547-4ca5-acdf-8eb7113868b1 started at 2021-05-15 01:44:42 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.667: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:26.667: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.667: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:46:26.667: INFO: pod-dc18f44a-4eff-4c02-9f90-6144d79f277b started at 2021-05-15 01:44:29 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.667: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:26.667: INFO: pod-bfa4b2b7-b6e7-46d4-9222-2bd34e874439 started at 2021-05-15 01:44:21 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.667: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:26.667: INFO: node-feature-discovery-worker-bw8zg started at 2021-05-15 00:18:56 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.667: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:46:26.667: INFO: collectd-mrzps started at 2021-05-15 00:19:22 +0000 UTC (0+3 container statuses recorded) May 15 01:46:26.667: INFO: Container collectd ready: true, restart count 0 May 15 01:46:26.667: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:46:26.667: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:46:26.667: INFO: hostexec-node1-lmdw4 started at 2021-05-15 01:46:14 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.667: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:26.667: INFO: pod-141a54ce-270d-46f4-a49c-b05dd96fbcdd started at 2021-05-15 01:44:28 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.667: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:26.667: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:26.667: INFO: Init container install-cni ready: true, restart count 0 May 15 01:46:26.667: INFO: Container kube-flannel ready: true, restart count 1 W0515 01:46:26.678208 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:26.715: INFO: Latency metrics for node node1 May 15 01:46:26.715: INFO: Logging node info for node node2 May 15 01:46:26.718: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 154137 0 2021-05-14 19:58:22 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-3052":"csi-mock-csi-mock-volumes-3052","csi-mock-csi-mock-volumes-8760":"csi-mock-csi-mock-volumes-8760"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:46 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:44:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}},"f:status":{"f:volumesAttached":{}}}} {kubelet Update v1 2021-05-15 01:44:49 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}},"f:volumesInUse":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:19 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:19 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:19 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:19 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-3052^4],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-3052^4,DevicePath:,},},Config:nil,},} May 15 01:46:26.718: INFO: Logging kubelet events for node node2 May 15 01:46:26.723: INFO: Logging pods the kubelet thinks is on node node2 May 15 01:46:26.736: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 15 01:46:26.736: INFO: Container nodereport ready: true, restart count 0 May 15 01:46:26.736: INFO: Container reconcile ready: true, restart count 0 May 15 01:46:26.736: INFO: pod-05c687ae-4432-4531-8186-a9b33485c6d8 started at 2021-05-15 01:46:23 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.736: INFO: Container test-container ready: false, restart count 0 May 15 01:46:26.736: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.736: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:46:26.736: INFO: pvc-volume-tester-jq8ps started at 2021-05-15 01:46:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.736: INFO: Container volume-tester ready: false, restart count 0 May 15 01:46:26.737: INFO: csi-mockplugin-0 started at 2021-05-15 01:44:36 +0000 UTC (0+3 container statuses recorded) May 15 01:46:26.737: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:46:26.737: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:46:26.737: INFO: Container mock ready: true, restart count 0 May 15 01:46:26.737: INFO: csi-mockplugin-attacher-0 started at 2021-05-15 01:44:36 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.737: INFO: Container csi-attacher ready: true, restart count 0 May 15 01:46:26.737: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.737: INFO: Container cmk-webhook ready: true, restart count 0 May 15 01:46:26.737: INFO: csi-mockplugin-0 started at 2021-05-15 01:46:19 +0000 UTC (0+3 container statuses recorded) May 15 01:46:26.737: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:46:26.737: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:46:26.737: INFO: Container mock ready: true, restart count 0 May 15 01:46:26.737: INFO: csi-mockplugin-attacher-0 started at 2021-05-15 01:46:19 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.737: INFO: Container csi-attacher ready: true, restart count 0 May 15 01:46:26.737: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 15 01:46:26.737: INFO: Container discover ready: false, restart count 0 May 15 01:46:26.737: INFO: Container init ready: false, restart count 0 May 15 01:46:26.737: INFO: Container install ready: false, restart count 0 May 15 01:46:26.737: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.737: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:26.737: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.737: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 15 01:46:26.737: INFO: pvc-volume-tester-8nftk started at 2021-05-15 01:44:47 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.737: INFO: Container volume-tester ready: true, restart count 0 May 15 01:46:26.737: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.737: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:46:26.737: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.737: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:46:26.737: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:46:26.737: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:26.737: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:26.737: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:26.737: INFO: Init container install-cni ready: true, restart count 1 May 15 01:46:26.737: INFO: Container kube-flannel ready: true, restart count 4 May 15 01:46:26.737: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 15 01:46:26.737: INFO: Container collectd ready: true, restart count 0 May 15 01:46:26.737: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:46:26.737: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:46:26.737: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 15 01:46:26.737: INFO: Container tas-controller ready: true, restart count 0 May 15 01:46:26.737: INFO: Container tas-extender ready: true, restart count 0 May 15 01:46:26.737: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 15 01:46:26.737: INFO: Container kube-sriovdp ready: true, restart count 0 W0515 01:46:26.750486 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:26.795: INFO: Latency metrics for node node2 May 15 01:46:26.795: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-1942" for this suite. • Failure [148.455 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: dir-bindmounted] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 Two pods mounting a local volume at the same time /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:244 should be able to write from pod1 and read from pod2 [It] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:245 May 15 01:46:26.314: Unexpected error: <*errors.errorString | 0xc00436c570>: { s: "pod \"pod-1812e695-95fc-4df1-8d69-0fbfa00893b8\" is not Running: timed out waiting for the condition", } pod "pod-1812e695-95fc-4df1-8d69-0fbfa00893b8" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:740 ------------------------------ {"msg":"FAILED [sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2","total":-1,"completed":0,"skipped":2,"failed":1,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2"]} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] EmptyDir volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:46:23.166: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename emptydir STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] when FSGroup is specified [LinuxOnly] [NodeFeature:FSGroup] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/empty_dir.go:47 [It] nonexistent volume subPath should have the correct mode and owner using FSGroup /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/empty_dir.go:60 STEP: Creating a pod to test emptydir subpath on tmpfs May 15 01:46:23.204: INFO: Waiting up to 5m0s for pod "pod-05c687ae-4432-4531-8186-a9b33485c6d8" in namespace "emptydir-3827" to be "Succeeded or Failed" May 15 01:46:23.210: INFO: Pod "pod-05c687ae-4432-4531-8186-a9b33485c6d8": Phase="Pending", Reason="", readiness=false. Elapsed: 5.643417ms May 15 01:46:25.213: INFO: Pod "pod-05c687ae-4432-4531-8186-a9b33485c6d8": Phase="Pending", Reason="", readiness=false. Elapsed: 2.008365929s May 15 01:46:27.215: INFO: Pod "pod-05c687ae-4432-4531-8186-a9b33485c6d8": Phase="Succeeded", Reason="", readiness=false. Elapsed: 4.011116097s STEP: Saw pod success May 15 01:46:27.216: INFO: Pod "pod-05c687ae-4432-4531-8186-a9b33485c6d8" satisfied condition "Succeeded or Failed" May 15 01:46:27.218: INFO: Trying to get logs from node node2 pod pod-05c687ae-4432-4531-8186-a9b33485c6d8 container test-container: STEP: delete the pod May 15 01:46:27.231: INFO: Waiting for pod pod-05c687ae-4432-4531-8186-a9b33485c6d8 to disappear May 15 01:46:27.233: INFO: Pod pod-05c687ae-4432-4531-8186-a9b33485c6d8 no longer exists [AfterEach] [sig-storage] EmptyDir volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:46:27.233: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "emptydir-3827" for this suite. • ------------------------------ {"msg":"PASSED [sig-storage] EmptyDir volumes when FSGroup is specified [LinuxOnly] [NodeFeature:FSGroup] nonexistent volume subPath should have the correct mode and owner using FSGroup","total":-1,"completed":2,"skipped":7,"failed":0} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:44:14.548: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: dir-bindmounted] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes May 15 01:44:26.594: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir /tmp/local-volume-test-8a1c4be0-2838-4253-919a-f2512bae7ec3 && mount --bind /tmp/local-volume-test-8a1c4be0-2838-4253-919a-f2512bae7ec3 /tmp/local-volume-test-8a1c4be0-2838-4253-919a-f2512bae7ec3] Namespace:persistent-local-volumes-test-1909 PodName:hostexec-node1-xrzq8 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:44:26.594: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:44:26.746: INFO: Creating a PV followed by a PVC May 15 01:44:26.752: INFO: Waiting for PV local-pv2vl4f to bind to PVC pvc-wpsn5 May 15 01:44:26.752: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-wpsn5] to have phase Bound May 15 01:44:26.754: INFO: PersistentVolumeClaim pvc-wpsn5 found but phase is Pending instead of Bound. May 15 01:44:28.757: INFO: PersistentVolumeClaim pvc-wpsn5 found and phase=Bound (2.00483351s) May 15 01:44:28.757: INFO: Waiting up to 3m0s for PersistentVolume local-pv2vl4f to have phase Bound May 15 01:44:28.759: INFO: PersistentVolume local-pv2vl4f found and phase=Bound (2.339839ms) [BeforeEach] One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:211 STEP: Creating pod1 STEP: Creating a pod May 15 01:46:28.782: FAIL: Unexpected error: <*errors.errorString | 0xc003f52820>: { s: "pod \"pod-141a54ce-270d-46f4-a49c-b05dd96fbcdd\" is not Running: timed out waiting for the condition", } pod "pod-141a54ce-270d-46f4-a49c-b05dd96fbcdd" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage.glob..func20.2.3.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 +0x119 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000e53800) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc000e53800) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc000e53800, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:223 STEP: Deleting pod1 STEP: Deleting pod pod-141a54ce-270d-46f4-a49c-b05dd96fbcdd in namespace persistent-local-volumes-test-1909 [AfterEach] [Volume type: dir-bindmounted] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:46:28.788: INFO: Deleting PersistentVolumeClaim "pvc-wpsn5" May 15 01:46:28.792: INFO: Deleting PersistentVolume "local-pv2vl4f" STEP: Removing the test directory May 15 01:46:28.797: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c umount /tmp/local-volume-test-8a1c4be0-2838-4253-919a-f2512bae7ec3 && rm -r /tmp/local-volume-test-8a1c4be0-2838-4253-919a-f2512bae7ec3] Namespace:persistent-local-volumes-test-1909 PodName:hostexec-node1-xrzq8 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:46:28.797: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "persistent-local-volumes-test-1909". STEP: Found 16 events. May 15 01:46:28.929: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for hostexec-node1-xrzq8: { } Scheduled: Successfully assigned persistent-local-volumes-test-1909/hostexec-node1-xrzq8 to node1 May 15 01:46:28.929: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-141a54ce-270d-46f4-a49c-b05dd96fbcdd: { } Scheduled: Successfully assigned persistent-local-volumes-test-1909/pod-141a54ce-270d-46f4-a49c-b05dd96fbcdd to node1 May 15 01:46:28.929: INFO: At 2021-05-15 01:44:17 +0000 UTC - event for hostexec-node1-xrzq8: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 15 01:46:28.929: INFO: At 2021-05-15 01:44:19 +0000 UTC - event for hostexec-node1-xrzq8: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 1.655733002s May 15 01:46:28.929: INFO: At 2021-05-15 01:44:19 +0000 UTC - event for hostexec-node1-xrzq8: {kubelet node1} Created: Created container agnhost-container May 15 01:46:28.929: INFO: At 2021-05-15 01:44:19 +0000 UTC - event for hostexec-node1-xrzq8: {kubelet node1} Started: Started container agnhost-container May 15 01:46:28.929: INFO: At 2021-05-15 01:44:26 +0000 UTC - event for pvc-wpsn5: {persistentvolume-controller } ProvisioningFailed: no volume plugin matched name: kubernetes.io/no-provisioner May 15 01:46:28.929: INFO: At 2021-05-15 01:44:31 +0000 UTC - event for pod-141a54ce-270d-46f4-a49c-b05dd96fbcdd: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:46:28.929: INFO: At 2021-05-15 01:44:31 +0000 UTC - event for pod-141a54ce-270d-46f4-a49c-b05dd96fbcdd: {multus } AddedInterface: Add eth0 [10.244.3.127/24] May 15 01:46:28.929: INFO: At 2021-05-15 01:44:32 +0000 UTC - event for pod-141a54ce-270d-46f4-a49c-b05dd96fbcdd: {kubelet node1} Failed: Error: ErrImagePull May 15 01:46:28.929: INFO: At 2021-05-15 01:44:32 +0000 UTC - event for pod-141a54ce-270d-46f4-a49c-b05dd96fbcdd: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:46:28.929: INFO: At 2021-05-15 01:44:34 +0000 UTC - event for pod-141a54ce-270d-46f4-a49c-b05dd96fbcdd: {kubelet node1} SandboxChanged: Pod sandbox changed, it will be killed and re-created. May 15 01:46:28.929: INFO: At 2021-05-15 01:44:36 +0000 UTC - event for pod-141a54ce-270d-46f4-a49c-b05dd96fbcdd: {multus } AddedInterface: Add eth0 [10.244.3.130/24] May 15 01:46:28.929: INFO: At 2021-05-15 01:44:36 +0000 UTC - event for pod-141a54ce-270d-46f4-a49c-b05dd96fbcdd: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 15 01:46:28.929: INFO: At 2021-05-15 01:44:36 +0000 UTC - event for pod-141a54ce-270d-46f4-a49c-b05dd96fbcdd: {kubelet node1} Failed: Error: ImagePullBackOff May 15 01:46:28.929: INFO: At 2021-05-15 01:44:41 +0000 UTC - event for pod-141a54ce-270d-46f4-a49c-b05dd96fbcdd: {multus } AddedInterface: Add eth0 [10.244.3.133/24] May 15 01:46:28.932: INFO: POD NODE PHASE GRACE CONDITIONS May 15 01:46:28.932: INFO: hostexec-node1-xrzq8 node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:14 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:20 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:20 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:14 +0000 UTC }] May 15 01:46:28.932: INFO: pod-141a54ce-270d-46f4-a49c-b05dd96fbcdd node1 Pending 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:28 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:28 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:28 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:28 +0000 UTC }] May 15 01:46:28.932: INFO: May 15 01:46:28.936: INFO: Logging node info for node master1 May 15 01:46:28.939: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 154243 0 2021-05-14 19:56:35 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:27 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:27 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:27 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:27 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[@ :],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:46:28.939: INFO: Logging kubelet events for node master1 May 15 01:46:28.944: INFO: Logging pods the kubelet thinks is on node master1 May 15 01:46:28.965: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:28.965: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:46:28.965: INFO: kubernetes-metrics-scraper-678c97765c-fswrn started at 2021-05-15 00:18:49 +0000 UTC (0+1 container statuses recorded) May 15 01:46:28.965: INFO: Container kubernetes-metrics-scraper ready: true, restart count 0 May 15 01:46:28.965: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:28.965: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:46:28.965: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:28.965: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:28.965: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 15 01:46:28.965: INFO: Container docker-registry ready: true, restart count 0 May 15 01:46:28.965: INFO: Container nginx ready: true, restart count 0 May 15 01:46:28.965: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 15 01:46:28.965: INFO: Container nfd-controller ready: true, restart count 0 May 15 01:46:28.965: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:46:28.965: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:46:28.965: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:28.965: INFO: Init container install-cni ready: true, restart count 0 May 15 01:46:28.965: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:46:28.965: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:46:28.965: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:28.965: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:28.965: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:28.965: INFO: Container kube-scheduler ready: true, restart count 0 W0515 01:46:28.978481 28 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:29.010: INFO: Latency metrics for node master1 May 15 01:46:29.010: INFO: Logging node info for node master2 May 15 01:46:29.012: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 154162 0 2021-05-14 19:57:04 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:25 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:25 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:25 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:25 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:46:29.012: INFO: Logging kubelet events for node master2 May 15 01:46:29.016: INFO: Logging pods the kubelet thinks is on node master2 May 15 01:46:29.026: INFO: prometheus-operator-5bb8cb9d8f-fqb87 started at 2021-05-15 00:18:49 +0000 UTC (0+2 container statuses recorded) May 15 01:46:29.026: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:29.026: INFO: Container prometheus-operator ready: true, restart count 0 May 15 01:46:29.026: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.026: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:46:29.026: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.026: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:46:29.026: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.026: INFO: Container kube-scheduler ready: true, restart count 2 May 15 01:46:29.026: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:29.026: INFO: Init container install-cni ready: true, restart count 0 May 15 01:46:29.026: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:46:29.026: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.026: INFO: Container autoscaler ready: true, restart count 2 May 15 01:46:29.026: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:46:29.026: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:29.026: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:29.026: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.026: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:46:29.026: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.026: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:29.026: INFO: coredns-7677f9bb54-96w24 started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.026: INFO: Container coredns ready: true, restart count 0 W0515 01:46:29.037022 28 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:29.061: INFO: Latency metrics for node master2 May 15 01:46:29.061: INFO: Logging node info for node master3 May 15 01:46:29.063: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 154124 0 2021-05-14 19:57:14 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234767872 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324603904 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:23 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:23 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:23 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:23 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:46:29.064: INFO: Logging kubelet events for node master3 May 15 01:46:29.067: INFO: Logging pods the kubelet thinks is on node master3 May 15 01:46:29.074: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.074: INFO: Container kube-controller-manager ready: true, restart count 3 May 15 01:46:29.074: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.074: INFO: Container kube-scheduler ready: true, restart count 3 May 15 01:46:29.074: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.074: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:46:29.074: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.074: INFO: Container kube-proxy ready: true, restart count 1 May 15 01:46:29.074: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:29.074: INFO: Init container install-cni ready: true, restart count 0 May 15 01:46:29.074: INFO: Container kube-flannel ready: true, restart count 2 May 15 01:46:29.074: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.074: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:29.075: INFO: coredns-7677f9bb54-rpj8c started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.075: INFO: Container coredns ready: true, restart count 0 May 15 01:46:29.075: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:46:29.075: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:29.075: INFO: Container node-exporter ready: true, restart count 0 W0515 01:46:29.087361 28 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:29.114: INFO: Latency metrics for node master3 May 15 01:46:29.114: INFO: Logging node info for node node1 May 15 01:46:29.117: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 154091 0 2021-05-14 19:58:21 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-7734":"csi-mock-csi-mock-volumes-7734"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:00 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:44:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:46:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:23 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:23 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:23 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:23 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[@ :],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:46:29.118: INFO: Logging kubelet events for node node1 May 15 01:46:29.122: INFO: Logging pods the kubelet thinks is on node node1 May 15 01:46:29.137: INFO: hostexec-node1-r82s6 started at 2021-05-15 01:46:27 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.138: INFO: Container agnhost-container ready: false, restart count 0 May 15 01:46:29.138: INFO: cmk-4s6dm started at 2021-05-15 00:18:54 +0000 UTC (0+2 container statuses recorded) May 15 01:46:29.138: INFO: Container nodereport ready: true, restart count 0 May 15 01:46:29.138: INFO: Container reconcile ready: true, restart count 0 May 15 01:46:29.138: INFO: hostexec-node1-5mrx2 started at 2021-05-15 01:44:12 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.138: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:29.138: INFO: hostexec-node1-dfp4d started at 2021-05-15 01:44:32 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.138: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:29.138: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.138: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:46:29.138: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4vxcc started at 2021-05-15 00:19:00 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.138: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:46:29.138: INFO: pod-38d6e6d5-eb01-4ca7-89a4-5da7e94c8500 started at 2021-05-15 01:44:25 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.138: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:29.138: INFO: pod-e1744dfa-1547-4ca5-acdf-8eb7113868b1 started at 2021-05-15 01:44:42 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.138: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:29.138: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.138: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:46:29.138: INFO: hostexec-node1-dzx4r started at 2021-05-15 01:43:58 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.138: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:29.138: INFO: pod-828316bc-b350-4ddb-9ce9-cec4d8c6cd45 started at 2021-05-15 01:44:13 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.138: INFO: Container write-pod ready: true, restart count 0 May 15 01:46:29.138: INFO: hostexec-node1-sv6vb started at 2021-05-15 01:46:26 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.138: INFO: Container agnhost-container ready: false, restart count 0 May 15 01:46:29.138: INFO: pod-dc18f44a-4eff-4c02-9f90-6144d79f277b started at 2021-05-15 01:44:29 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.138: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:29.138: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:29.138: INFO: Init container install-cni ready: true, restart count 0 May 15 01:46:29.138: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:46:29.138: INFO: node-feature-discovery-worker-bw8zg started at 2021-05-15 00:18:56 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.138: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:46:29.138: INFO: collectd-mrzps started at 2021-05-15 00:19:22 +0000 UTC (0+3 container statuses recorded) May 15 01:46:29.138: INFO: Container collectd ready: true, restart count 0 May 15 01:46:29.138: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:46:29.138: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:46:29.138: INFO: hostexec-node1-lmdw4 started at 2021-05-15 01:46:14 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.138: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:29.138: INFO: pod-141a54ce-270d-46f4-a49c-b05dd96fbcdd started at 2021-05-15 01:44:28 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.138: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:29.138: INFO: pod-604ae13e-a2a4-44cd-87c9-c2aa1f39edc0 started at 2021-05-15 01:46:28 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.138: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:29.138: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.138: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:29.138: INFO: prometheus-k8s-0 started at 2021-05-15 00:19:01 +0000 UTC (0+5 container statuses recorded) May 15 01:46:29.138: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 15 01:46:29.138: INFO: Container grafana ready: true, restart count 0 May 15 01:46:29.138: INFO: Container prometheus ready: true, restart count 26 May 15 01:46:29.138: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 15 01:46:29.138: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 15 01:46:29.138: INFO: pod-1812e695-95fc-4df1-8d69-0fbfa00893b8 started at 2021-05-15 01:44:26 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.138: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:29.138: INFO: hostexec-node1-5fw6h started at 2021-05-15 01:46:22 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.138: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:29.138: INFO: hostexec-node1-zpl46 started at 2021-05-15 01:44:13 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.138: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:29.138: INFO: node-exporter-flvqz started at 2021-05-15 00:18:55 +0000 UTC (0+2 container statuses recorded) May 15 01:46:29.138: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:29.138: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:29.138: INFO: hostexec-node1-xrzq8 started at 2021-05-15 01:44:14 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.138: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:29.138: INFO: pod-c61f2051-9957-419e-995d-c3ed08d36192 started at 2021-05-15 01:46:20 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.138: INFO: Container write-pod ready: false, restart count 0 W0515 01:46:29.151405 28 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:29.199: INFO: Latency metrics for node node1 May 15 01:46:29.199: INFO: Logging node info for node node2 May 15 01:46:29.202: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 154137 0 2021-05-14 19:58:22 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-3052":"csi-mock-csi-mock-volumes-3052","csi-mock-csi-mock-volumes-8760":"csi-mock-csi-mock-volumes-8760"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:46 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:44:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}},"f:status":{"f:volumesAttached":{}}}} {kubelet Update v1 2021-05-15 01:44:49 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}},"f:volumesInUse":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:19 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:19 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:19 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:19 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-3052^4],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-3052^4,DevicePath:,},},Config:nil,},} May 15 01:46:29.202: INFO: Logging kubelet events for node node2 May 15 01:46:29.207: INFO: Logging pods the kubelet thinks is on node node2 May 15 01:46:29.221: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:29.221: INFO: Init container install-cni ready: true, restart count 1 May 15 01:46:29.221: INFO: Container kube-flannel ready: true, restart count 4 May 15 01:46:29.221: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:46:29.221: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:29.221: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:29.221: INFO: pod-e7471185-34e8-415a-9615-7ae76ade89b8 started at 2021-05-15 01:46:26 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.221: INFO: Container test-container ready: false, restart count 0 May 15 01:46:29.221: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 15 01:46:29.221: INFO: Container tas-controller ready: true, restart count 0 May 15 01:46:29.221: INFO: Container tas-extender ready: true, restart count 0 May 15 01:46:29.221: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 15 01:46:29.221: INFO: Container collectd ready: true, restart count 0 May 15 01:46:29.221: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:46:29.221: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:46:29.221: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.221: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:46:29.221: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.221: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:46:29.221: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 15 01:46:29.221: INFO: Container nodereport ready: true, restart count 0 May 15 01:46:29.221: INFO: Container reconcile ready: true, restart count 0 May 15 01:46:29.221: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.221: INFO: Container cmk-webhook ready: true, restart count 0 May 15 01:46:29.221: INFO: pvc-volume-tester-jq8ps started at 2021-05-15 01:46:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.221: INFO: Container volume-tester ready: true, restart count 0 May 15 01:46:29.221: INFO: csi-mockplugin-0 started at 2021-05-15 01:44:36 +0000 UTC (0+3 container statuses recorded) May 15 01:46:29.221: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:46:29.221: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:46:29.221: INFO: Container mock ready: true, restart count 0 May 15 01:46:29.221: INFO: csi-mockplugin-attacher-0 started at 2021-05-15 01:44:36 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.221: INFO: Container csi-attacher ready: true, restart count 0 May 15 01:46:29.221: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 15 01:46:29.221: INFO: Container discover ready: false, restart count 0 May 15 01:46:29.221: INFO: Container init ready: false, restart count 0 May 15 01:46:29.221: INFO: Container install ready: false, restart count 0 May 15 01:46:29.221: INFO: csi-mockplugin-0 started at 2021-05-15 01:46:19 +0000 UTC (0+3 container statuses recorded) May 15 01:46:29.221: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:46:29.221: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:46:29.221: INFO: Container mock ready: true, restart count 0 May 15 01:46:29.221: INFO: csi-mockplugin-attacher-0 started at 2021-05-15 01:46:19 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.221: INFO: Container csi-attacher ready: true, restart count 0 May 15 01:46:29.221: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.221: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:46:29.221: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.221: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:29.221: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.221: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 15 01:46:29.221: INFO: pvc-volume-tester-8nftk started at 2021-05-15 01:44:47 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.221: INFO: Container volume-tester ready: true, restart count 0 May 15 01:46:29.221: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:46:29.221: INFO: Container nginx-proxy ready: true, restart count 2 W0515 01:46:29.232035 28 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:29.275: INFO: Latency metrics for node node2 May 15 01:46:29.275: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-1909" for this suite. • Failure in Spec Setup (BeforeEach) [134.735 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: dir-bindmounted] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:205 should be able to mount volume and read from pod1 [BeforeEach] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:228 May 15 01:46:28.782: Unexpected error: <*errors.errorString | 0xc003f52820>: { s: "pod \"pod-141a54ce-270d-46f4-a49c-b05dd96fbcdd\" is not Running: timed out waiting for the condition", } pod "pod-141a54ce-270d-46f4-a49c-b05dd96fbcdd" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 ------------------------------ {"msg":"FAILED [sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] One pod requesting one prebound PVC should be able to mount volume and read from pod1","total":-1,"completed":2,"skipped":505,"failed":1,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] One pod requesting one prebound PVC should be able to mount volume and read from pod1"]} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:44:13.266: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: blockfswithoutformat] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes STEP: Creating block device on node "node1" using path "/tmp/local-volume-test-a855db8e-c439-419c-af4d-ba514c03d4e8" May 15 01:44:23.310: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-a855db8e-c439-419c-af4d-ba514c03d4e8 && dd if=/dev/zero of=/tmp/local-volume-test-a855db8e-c439-419c-af4d-ba514c03d4e8/file bs=4096 count=5120 && losetup -f /tmp/local-volume-test-a855db8e-c439-419c-af4d-ba514c03d4e8/file] Namespace:persistent-local-volumes-test-9787 PodName:hostexec-node1-zpl46 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:44:23.310: INFO: >>> kubeConfig: /root/.kube/config May 15 01:44:23.446: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c E2E_LOOP_DEV=$(losetup | grep /tmp/local-volume-test-a855db8e-c439-419c-af4d-ba514c03d4e8/file | awk '{ print $1 }') 2>&1 > /dev/null && echo ${E2E_LOOP_DEV}] Namespace:persistent-local-volumes-test-9787 PodName:hostexec-node1-zpl46 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:44:23.446: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:44:23.562: INFO: Creating a PV followed by a PVC May 15 01:44:23.569: INFO: Waiting for PV local-pv82vbm to bind to PVC pvc-c6k4b May 15 01:44:23.569: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-c6k4b] to have phase Bound May 15 01:44:23.572: INFO: PersistentVolumeClaim pvc-c6k4b found but phase is Pending instead of Bound. May 15 01:44:25.575: INFO: PersistentVolumeClaim pvc-c6k4b found but phase is Pending instead of Bound. May 15 01:44:27.578: INFO: PersistentVolumeClaim pvc-c6k4b found but phase is Pending instead of Bound. May 15 01:44:29.582: INFO: PersistentVolumeClaim pvc-c6k4b found and phase=Bound (6.013288686s) May 15 01:44:29.582: INFO: Waiting up to 3m0s for PersistentVolume local-pv82vbm to have phase Bound May 15 01:44:29.584: INFO: PersistentVolume local-pv82vbm found and phase=Bound (2.169313ms) [BeforeEach] One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:211 STEP: Creating pod1 STEP: Creating a pod May 15 01:46:29.608: FAIL: Unexpected error: <*errors.errorString | 0xc00500c4e0>: { s: "pod \"pod-dc18f44a-4eff-4c02-9f90-6144d79f277b\" is not Running: timed out waiting for the condition", } pod "pod-dc18f44a-4eff-4c02-9f90-6144d79f277b" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage.glob..func20.2.3.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 +0x119 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc001d81980) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc001d81980) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc001d81980, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:223 STEP: Deleting pod1 STEP: Deleting pod pod-dc18f44a-4eff-4c02-9f90-6144d79f277b in namespace persistent-local-volumes-test-9787 [AfterEach] [Volume type: blockfswithoutformat] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:46:29.614: INFO: Deleting PersistentVolumeClaim "pvc-c6k4b" May 15 01:46:29.618: INFO: Deleting PersistentVolume "local-pv82vbm" May 15 01:46:29.621: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c E2E_LOOP_DEV=$(losetup | grep /tmp/local-volume-test-a855db8e-c439-419c-af4d-ba514c03d4e8/file | awk '{ print $1 }') 2>&1 > /dev/null && echo ${E2E_LOOP_DEV}] Namespace:persistent-local-volumes-test-9787 PodName:hostexec-node1-zpl46 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:46:29.621: INFO: >>> kubeConfig: /root/.kube/config STEP: Tear down block device "/dev/loop1" on node "node1" at path /tmp/local-volume-test-a855db8e-c439-419c-af4d-ba514c03d4e8/file May 15 01:46:29.897: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c losetup -d /dev/loop1] Namespace:persistent-local-volumes-test-9787 PodName:hostexec-node1-zpl46 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:46:29.897: INFO: >>> kubeConfig: /root/.kube/config STEP: Removing the test directory /tmp/local-volume-test-a855db8e-c439-419c-af4d-ba514c03d4e8 May 15 01:46:30.343: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-a855db8e-c439-419c-af4d-ba514c03d4e8] Namespace:persistent-local-volumes-test-9787 PodName:hostexec-node1-zpl46 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:46:30.343: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "persistent-local-volumes-test-9787". STEP: Found 18 events. May 15 01:46:30.618: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for hostexec-node1-zpl46: { } Scheduled: Successfully assigned persistent-local-volumes-test-9787/hostexec-node1-zpl46 to node1 May 15 01:46:30.618: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-dc18f44a-4eff-4c02-9f90-6144d79f277b: { } Scheduled: Successfully assigned persistent-local-volumes-test-9787/pod-dc18f44a-4eff-4c02-9f90-6144d79f277b to node1 May 15 01:46:30.618: INFO: At 2021-05-15 01:44:16 +0000 UTC - event for hostexec-node1-zpl46: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 15 01:46:30.618: INFO: At 2021-05-15 01:44:16 +0000 UTC - event for hostexec-node1-zpl46: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 517.666915ms May 15 01:46:30.618: INFO: At 2021-05-15 01:44:17 +0000 UTC - event for hostexec-node1-zpl46: {kubelet node1} Created: Created container agnhost-container May 15 01:46:30.618: INFO: At 2021-05-15 01:44:17 +0000 UTC - event for hostexec-node1-zpl46: {kubelet node1} Started: Started container agnhost-container May 15 01:46:30.618: INFO: At 2021-05-15 01:44:23 +0000 UTC - event for pvc-c6k4b: {persistentvolume-controller } ProvisioningFailed: no volume plugin matched name: kubernetes.io/no-provisioner May 15 01:46:30.618: INFO: At 2021-05-15 01:44:34 +0000 UTC - event for pod-dc18f44a-4eff-4c02-9f90-6144d79f277b: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:46:30.618: INFO: At 2021-05-15 01:44:34 +0000 UTC - event for pod-dc18f44a-4eff-4c02-9f90-6144d79f277b: {multus } AddedInterface: Add eth0 [10.244.3.129/24] May 15 01:46:30.618: INFO: At 2021-05-15 01:44:35 +0000 UTC - event for pod-dc18f44a-4eff-4c02-9f90-6144d79f277b: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:46:30.618: INFO: At 2021-05-15 01:44:35 +0000 UTC - event for pod-dc18f44a-4eff-4c02-9f90-6144d79f277b: {kubelet node1} Failed: Error: ErrImagePull May 15 01:46:30.618: INFO: At 2021-05-15 01:44:36 +0000 UTC - event for pod-dc18f44a-4eff-4c02-9f90-6144d79f277b: {kubelet node1} SandboxChanged: Pod sandbox changed, it will be killed and re-created. May 15 01:46:30.618: INFO: At 2021-05-15 01:44:39 +0000 UTC - event for pod-dc18f44a-4eff-4c02-9f90-6144d79f277b: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 15 01:46:30.618: INFO: At 2021-05-15 01:44:39 +0000 UTC - event for pod-dc18f44a-4eff-4c02-9f90-6144d79f277b: {kubelet node1} Failed: Error: ImagePullBackOff May 15 01:46:30.618: INFO: At 2021-05-15 01:44:39 +0000 UTC - event for pod-dc18f44a-4eff-4c02-9f90-6144d79f277b: {multus } AddedInterface: Add eth0 [10.244.3.132/24] May 15 01:46:30.618: INFO: At 2021-05-15 01:44:43 +0000 UTC - event for pod-dc18f44a-4eff-4c02-9f90-6144d79f277b: {multus } AddedInterface: Add eth0 [10.244.3.136/24] May 15 01:46:30.618: INFO: At 2021-05-15 01:44:47 +0000 UTC - event for pod-dc18f44a-4eff-4c02-9f90-6144d79f277b: {multus } AddedInterface: Add eth0 [10.244.3.138/24] May 15 01:46:30.618: INFO: At 2021-05-15 01:44:50 +0000 UTC - event for pod-dc18f44a-4eff-4c02-9f90-6144d79f277b: {multus } AddedInterface: Add eth0 [10.244.3.139/24] May 15 01:46:30.621: INFO: POD NODE PHASE GRACE CONDITIONS May 15 01:46:30.621: INFO: hostexec-node1-zpl46 node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:17 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:17 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:13 +0000 UTC }] May 15 01:46:30.621: INFO: pod-dc18f44a-4eff-4c02-9f90-6144d79f277b node1 Pending 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:29 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:29 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:29 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:29 +0000 UTC }] May 15 01:46:30.621: INFO: May 15 01:46:30.625: INFO: Logging node info for node master1 May 15 01:46:30.628: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 154243 0 2021-05-14 19:56:35 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:27 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:27 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:27 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:27 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[@ :],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:46:30.628: INFO: Logging kubelet events for node master1 May 15 01:46:30.633: INFO: Logging pods the kubelet thinks is on node master1 May 15 01:46:30.643: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.643: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:46:30.643: INFO: kubernetes-metrics-scraper-678c97765c-fswrn started at 2021-05-15 00:18:49 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.643: INFO: Container kubernetes-metrics-scraper ready: true, restart count 0 May 15 01:46:30.643: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.643: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:46:30.643: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.643: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:30.643: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 15 01:46:30.643: INFO: Container docker-registry ready: true, restart count 0 May 15 01:46:30.643: INFO: Container nginx ready: true, restart count 0 May 15 01:46:30.643: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.643: INFO: Container nfd-controller ready: true, restart count 0 May 15 01:46:30.643: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.643: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:46:30.643: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:30.643: INFO: Init container install-cni ready: true, restart count 0 May 15 01:46:30.643: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:46:30.643: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:46:30.643: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:30.643: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:30.643: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.643: INFO: Container kube-scheduler ready: true, restart count 0 W0515 01:46:30.656044 21 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:30.683: INFO: Latency metrics for node master1 May 15 01:46:30.683: INFO: Logging node info for node master2 May 15 01:46:30.686: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 154162 0 2021-05-14 19:57:04 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:25 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:25 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:25 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:25 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:46:30.686: INFO: Logging kubelet events for node master2 May 15 01:46:30.690: INFO: Logging pods the kubelet thinks is on node master2 May 15 01:46:30.699: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.699: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:46:30.699: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.699: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:30.699: INFO: coredns-7677f9bb54-96w24 started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.699: INFO: Container coredns ready: true, restart count 0 May 15 01:46:30.699: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:46:30.699: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:30.699: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:30.699: INFO: prometheus-operator-5bb8cb9d8f-fqb87 started at 2021-05-15 00:18:49 +0000 UTC (0+2 container statuses recorded) May 15 01:46:30.699: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:30.699: INFO: Container prometheus-operator ready: true, restart count 0 May 15 01:46:30.699: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.699: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:46:30.699: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.699: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:46:30.699: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.699: INFO: Container kube-scheduler ready: true, restart count 2 May 15 01:46:30.699: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:30.699: INFO: Init container install-cni ready: true, restart count 0 May 15 01:46:30.699: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:46:30.699: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.699: INFO: Container autoscaler ready: true, restart count 2 W0515 01:46:30.710424 21 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:30.736: INFO: Latency metrics for node master2 May 15 01:46:30.736: INFO: Logging node info for node master3 May 15 01:46:30.739: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 154124 0 2021-05-14 19:57:14 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234767872 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324603904 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:23 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:23 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:23 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:23 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:46:30.740: INFO: Logging kubelet events for node master3 May 15 01:46:30.743: INFO: Logging pods the kubelet thinks is on node master3 May 15 01:46:30.751: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.751: INFO: Container kube-scheduler ready: true, restart count 3 May 15 01:46:30.751: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.751: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:46:30.751: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.751: INFO: Container kube-proxy ready: true, restart count 1 May 15 01:46:30.751: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:30.751: INFO: Init container install-cni ready: true, restart count 0 May 15 01:46:30.751: INFO: Container kube-flannel ready: true, restart count 2 May 15 01:46:30.751: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.751: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:30.751: INFO: coredns-7677f9bb54-rpj8c started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.751: INFO: Container coredns ready: true, restart count 0 May 15 01:46:30.751: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:46:30.751: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:30.751: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:30.751: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.751: INFO: Container kube-controller-manager ready: true, restart count 3 W0515 01:46:30.763154 21 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:30.787: INFO: Latency metrics for node master3 May 15 01:46:30.788: INFO: Logging node info for node node1 May 15 01:46:30.791: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 154091 0 2021-05-14 19:58:21 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-7734":"csi-mock-csi-mock-volumes-7734"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:00 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:44:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:46:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:23 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:23 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:23 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:23 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[@ :],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:46:30.791: INFO: Logging kubelet events for node node1 May 15 01:46:30.796: INFO: Logging pods the kubelet thinks is on node node1 May 15 01:46:30.814: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.814: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:30.814: INFO: prometheus-k8s-0 started at 2021-05-15 00:19:01 +0000 UTC (0+5 container statuses recorded) May 15 01:46:30.814: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 15 01:46:30.814: INFO: Container grafana ready: true, restart count 0 May 15 01:46:30.814: INFO: Container prometheus ready: true, restart count 26 May 15 01:46:30.814: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 15 01:46:30.814: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 15 01:46:30.814: INFO: pod-1812e695-95fc-4df1-8d69-0fbfa00893b8 started at 2021-05-15 01:44:26 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.814: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:30.814: INFO: hostexec-node1-5fw6h started at 2021-05-15 01:46:22 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.814: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:30.814: INFO: hostexec-node1-zpl46 started at 2021-05-15 01:44:13 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.814: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:30.814: INFO: node-exporter-flvqz started at 2021-05-15 00:18:55 +0000 UTC (0+2 container statuses recorded) May 15 01:46:30.814: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:30.814: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:30.814: INFO: hostexec-node1-xrzq8 started at 2021-05-15 01:44:14 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.814: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:30.814: INFO: pod-c61f2051-9957-419e-995d-c3ed08d36192 started at 2021-05-15 01:46:20 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.814: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:30.814: INFO: hostexec-node1-r82s6 started at 2021-05-15 01:46:27 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.814: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:30.814: INFO: cmk-4s6dm started at 2021-05-15 00:18:54 +0000 UTC (0+2 container statuses recorded) May 15 01:46:30.814: INFO: Container nodereport ready: true, restart count 0 May 15 01:46:30.814: INFO: Container reconcile ready: true, restart count 0 May 15 01:46:30.814: INFO: hostexec-node1-5mrx2 started at 2021-05-15 01:44:12 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.814: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:30.814: INFO: hostexec-node1-dfp4d started at 2021-05-15 01:44:32 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.814: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:30.814: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.814: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:46:30.814: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4vxcc started at 2021-05-15 00:19:00 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.814: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:46:30.814: INFO: pod-e1744dfa-1547-4ca5-acdf-8eb7113868b1 started at 2021-05-15 01:44:42 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.814: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:30.814: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.814: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:46:30.814: INFO: hostexec-node1-dzx4r started at 2021-05-15 01:43:58 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.814: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:30.814: INFO: pod-828316bc-b350-4ddb-9ce9-cec4d8c6cd45 started at 2021-05-15 01:44:13 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.814: INFO: Container write-pod ready: true, restart count 0 May 15 01:46:30.814: INFO: hostexec-node1-sv6vb started at 2021-05-15 01:46:26 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.814: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:30.814: INFO: pod-dc18f44a-4eff-4c02-9f90-6144d79f277b started at 2021-05-15 01:44:29 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.814: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:30.814: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:30.814: INFO: Init container install-cni ready: true, restart count 0 May 15 01:46:30.814: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:46:30.814: INFO: node-feature-discovery-worker-bw8zg started at 2021-05-15 00:18:56 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.814: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:46:30.814: INFO: collectd-mrzps started at 2021-05-15 00:19:22 +0000 UTC (0+3 container statuses recorded) May 15 01:46:30.814: INFO: Container collectd ready: true, restart count 0 May 15 01:46:30.814: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:46:30.814: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:46:30.814: INFO: hostexec-node1-lmdw4 started at 2021-05-15 01:46:14 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.814: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:30.814: INFO: pod-141a54ce-270d-46f4-a49c-b05dd96fbcdd started at 2021-05-15 01:44:28 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.814: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:30.814: INFO: pod-604ae13e-a2a4-44cd-87c9-c2aa1f39edc0 started at 2021-05-15 01:46:28 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.814: INFO: Container write-pod ready: false, restart count 0 W0515 01:46:30.826433 21 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:30.874: INFO: Latency metrics for node node1 May 15 01:46:30.874: INFO: Logging node info for node node2 May 15 01:46:30.876: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 154294 0 2021-05-14 19:58:22 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-3052":"csi-mock-csi-mock-volumes-3052","csi-mock-csi-mock-volumes-8760":"csi-mock-csi-mock-volumes-8760"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:46 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:44:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}},"f:status":{"f:volumesAttached":{}}}} {kubelet Update v1 2021-05-15 01:44:49 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}},"f:volumesInUse":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:29 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:29 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:29 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:29 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-3052^4],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-3052^4,DevicePath:,},},Config:nil,},} May 15 01:46:30.877: INFO: Logging kubelet events for node node2 May 15 01:46:30.882: INFO: Logging pods the kubelet thinks is on node node2 May 15 01:46:30.897: INFO: pod-e7471185-34e8-415a-9615-7ae76ade89b8 started at 2021-05-15 01:46:26 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.897: INFO: Container test-container ready: false, restart count 0 May 15 01:46:30.897: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:30.897: INFO: Init container install-cni ready: true, restart count 1 May 15 01:46:30.897: INFO: Container kube-flannel ready: true, restart count 4 May 15 01:46:30.897: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:46:30.897: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:30.897: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:30.897: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 15 01:46:30.897: INFO: Container tas-controller ready: true, restart count 0 May 15 01:46:30.897: INFO: Container tas-extender ready: true, restart count 0 May 15 01:46:30.897: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 15 01:46:30.897: INFO: Container collectd ready: true, restart count 0 May 15 01:46:30.897: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:46:30.897: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:46:30.897: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.897: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:46:30.897: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.897: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:46:30.897: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 15 01:46:30.897: INFO: Container nodereport ready: true, restart count 0 May 15 01:46:30.897: INFO: Container reconcile ready: true, restart count 0 May 15 01:46:30.897: INFO: csi-mockplugin-0 started at 2021-05-15 01:44:36 +0000 UTC (0+3 container statuses recorded) May 15 01:46:30.897: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:46:30.897: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:46:30.897: INFO: Container mock ready: true, restart count 0 May 15 01:46:30.897: INFO: csi-mockplugin-attacher-0 started at 2021-05-15 01:44:36 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.897: INFO: Container csi-attacher ready: true, restart count 0 May 15 01:46:30.897: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.897: INFO: Container cmk-webhook ready: true, restart count 0 May 15 01:46:30.897: INFO: pvc-volume-tester-jq8ps started at 2021-05-15 01:46:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.897: INFO: Container volume-tester ready: true, restart count 0 May 15 01:46:30.897: INFO: csi-mockplugin-attacher-0 started at 2021-05-15 01:46:19 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.897: INFO: Container csi-attacher ready: true, restart count 0 May 15 01:46:30.897: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 15 01:46:30.897: INFO: Container discover ready: false, restart count 0 May 15 01:46:30.897: INFO: Container init ready: false, restart count 0 May 15 01:46:30.897: INFO: Container install ready: false, restart count 0 May 15 01:46:30.897: INFO: csi-mockplugin-0 started at 2021-05-15 01:46:19 +0000 UTC (0+3 container statuses recorded) May 15 01:46:30.897: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:46:30.897: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:46:30.897: INFO: Container mock ready: true, restart count 0 May 15 01:46:30.897: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.897: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 15 01:46:30.897: INFO: pvc-volume-tester-8nftk started at 2021-05-15 01:44:47 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.897: INFO: Container volume-tester ready: true, restart count 0 May 15 01:46:30.897: INFO: pod-projected-configmaps-e3fee644-50ea-41ff-8391-5b20432a3b17 started at 2021-05-15 01:46:29 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.897: INFO: Container projected-configmap-volume-test ready: false, restart count 0 May 15 01:46:30.897: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.897: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:46:30.897: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.897: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:30.897: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:46:30.897: INFO: Container nginx-proxy ready: true, restart count 2 W0515 01:46:30.908030 21 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:30.945: INFO: Latency metrics for node node2 May 15 01:46:30.945: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-9787" for this suite. • Failure in Spec Setup (BeforeEach) [137.687 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: blockfswithoutformat] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:205 should be able to mount volume and read from pod1 [BeforeEach] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:228 May 15 01:46:29.608: Unexpected error: <*errors.errorString | 0xc00500c4e0>: { s: "pod \"pod-dc18f44a-4eff-4c02-9f90-6144d79f277b\" is not Running: timed out waiting for the condition", } pod "pod-dc18f44a-4eff-4c02-9f90-6144d79f277b" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 ------------------------------ {"msg":"FAILED [sig-storage] PersistentVolumes-local [Volume type: blockfswithoutformat] One pod requesting one prebound PVC should be able to mount volume and read from pod1","total":-1,"completed":1,"skipped":42,"failed":1,"failures":["[sig-storage] PersistentVolumes-local [Volume type: blockfswithoutformat] One pod requesting one prebound PVC should be able to mount volume and read from pod1"]} SSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] EmptyDir volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:46:26.938: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename emptydir STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] when FSGroup is specified [LinuxOnly] [NodeFeature:FSGroup] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/empty_dir.go:47 [It] volume on default medium should have the correct mode using FSGroup /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/empty_dir.go:68 STEP: Creating a pod to test emptydir volume type on node default medium May 15 01:46:26.970: INFO: Waiting up to 5m0s for pod "pod-e7471185-34e8-415a-9615-7ae76ade89b8" in namespace "emptydir-4354" to be "Succeeded or Failed" May 15 01:46:26.973: INFO: Pod "pod-e7471185-34e8-415a-9615-7ae76ade89b8": Phase="Pending", Reason="", readiness=false. Elapsed: 2.460897ms May 15 01:46:28.975: INFO: Pod "pod-e7471185-34e8-415a-9615-7ae76ade89b8": Phase="Pending", Reason="", readiness=false. Elapsed: 2.005021451s May 15 01:46:30.978: INFO: Pod "pod-e7471185-34e8-415a-9615-7ae76ade89b8": Phase="Succeeded", Reason="", readiness=false. Elapsed: 4.007699982s STEP: Saw pod success May 15 01:46:30.978: INFO: Pod "pod-e7471185-34e8-415a-9615-7ae76ade89b8" satisfied condition "Succeeded or Failed" May 15 01:46:30.980: INFO: Trying to get logs from node node2 pod pod-e7471185-34e8-415a-9615-7ae76ade89b8 container test-container: STEP: delete the pod May 15 01:46:31.002: INFO: Waiting for pod pod-e7471185-34e8-415a-9615-7ae76ade89b8 to disappear May 15 01:46:31.004: INFO: Pod pod-e7471185-34e8-415a-9615-7ae76ade89b8 no longer exists [AfterEach] [sig-storage] EmptyDir volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:46:31.004: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "emptydir-4354" for this suite. •S ------------------------------ {"msg":"PASSED [sig-storage] EmptyDir volumes when FSGroup is specified [LinuxOnly] [NodeFeature:FSGroup] volume on default medium should have the correct mode using FSGroup","total":-1,"completed":1,"skipped":66,"failed":1,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2"]} SSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] Projected configMap /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:46:29.418: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename projected STEP: Waiting for a default service account to be provisioned in namespace [It] should be consumable from pods in volume as non-root with FSGroup [LinuxOnly] [NodeFeature:FSGroup] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/projected_configmap.go:75 STEP: Creating configMap with name projected-configmap-test-volume-ec309e0d-27c7-4644-b66a-06848dc6c6e0 STEP: Creating a pod to test consume configMaps May 15 01:46:29.453: INFO: Waiting up to 5m0s for pod "pod-projected-configmaps-e3fee644-50ea-41ff-8391-5b20432a3b17" in namespace "projected-9692" to be "Succeeded or Failed" May 15 01:46:29.457: INFO: Pod "pod-projected-configmaps-e3fee644-50ea-41ff-8391-5b20432a3b17": Phase="Pending", Reason="", readiness=false. Elapsed: 3.797251ms May 15 01:46:31.460: INFO: Pod "pod-projected-configmaps-e3fee644-50ea-41ff-8391-5b20432a3b17": Phase="Pending", Reason="", readiness=false. Elapsed: 2.006919768s May 15 01:46:33.463: INFO: Pod "pod-projected-configmaps-e3fee644-50ea-41ff-8391-5b20432a3b17": Phase="Succeeded", Reason="", readiness=false. Elapsed: 4.009750769s STEP: Saw pod success May 15 01:46:33.463: INFO: Pod "pod-projected-configmaps-e3fee644-50ea-41ff-8391-5b20432a3b17" satisfied condition "Succeeded or Failed" May 15 01:46:33.465: INFO: Trying to get logs from node node2 pod pod-projected-configmaps-e3fee644-50ea-41ff-8391-5b20432a3b17 container projected-configmap-volume-test: STEP: delete the pod May 15 01:46:33.482: INFO: Waiting for pod pod-projected-configmaps-e3fee644-50ea-41ff-8391-5b20432a3b17 to disappear May 15 01:46:33.484: INFO: Pod pod-projected-configmaps-e3fee644-50ea-41ff-8391-5b20432a3b17 no longer exists [AfterEach] [sig-storage] Projected configMap /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:46:33.484: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "projected-9692" for this suite. • ------------------------------ {"msg":"PASSED [sig-storage] Projected configMap should be consumable from pods in volume as non-root with FSGroup [LinuxOnly] [NodeFeature:FSGroup]","total":-1,"completed":3,"skipped":569,"failed":1,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] One pod requesting one prebound PVC should be able to mount volume and read from pod1"]} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:44:32.444: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: dir-bindmounted] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes May 15 01:44:40.505: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir /tmp/local-volume-test-d1912c93-c852-4f94-8ac0-95cab3726d75 && mount --bind /tmp/local-volume-test-d1912c93-c852-4f94-8ac0-95cab3726d75 /tmp/local-volume-test-d1912c93-c852-4f94-8ac0-95cab3726d75] Namespace:persistent-local-volumes-test-2962 PodName:hostexec-node1-dfp4d ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:44:40.505: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:44:40.820: INFO: Creating a PV followed by a PVC May 15 01:44:40.828: INFO: Waiting for PV local-pvdh5c6 to bind to PVC pvc-4tcn9 May 15 01:44:40.828: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-4tcn9] to have phase Bound May 15 01:44:40.830: INFO: PersistentVolumeClaim pvc-4tcn9 found but phase is Pending instead of Bound. May 15 01:44:42.833: INFO: PersistentVolumeClaim pvc-4tcn9 found and phase=Bound (2.00550619s) May 15 01:44:42.833: INFO: Waiting up to 3m0s for PersistentVolume local-pvdh5c6 to have phase Bound May 15 01:44:42.836: INFO: PersistentVolume local-pvdh5c6 found and phase=Bound (2.473096ms) [BeforeEach] One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:211 STEP: Creating pod1 STEP: Creating a pod May 15 01:46:42.861: FAIL: Unexpected error: <*errors.errorString | 0xc002c6c350>: { s: "pod \"pod-e1744dfa-1547-4ca5-acdf-8eb7113868b1\" is not Running: timed out waiting for the condition", } pod "pod-e1744dfa-1547-4ca5-acdf-8eb7113868b1" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage.glob..func20.2.3.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 +0x119 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc00362b080) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc00362b080) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc00362b080, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:223 STEP: Deleting pod1 STEP: Deleting pod pod-e1744dfa-1547-4ca5-acdf-8eb7113868b1 in namespace persistent-local-volumes-test-2962 [AfterEach] [Volume type: dir-bindmounted] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:46:42.869: INFO: Deleting PersistentVolumeClaim "pvc-4tcn9" May 15 01:46:42.873: INFO: Deleting PersistentVolume "local-pvdh5c6" STEP: Removing the test directory May 15 01:46:42.877: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c umount /tmp/local-volume-test-d1912c93-c852-4f94-8ac0-95cab3726d75 && rm -r /tmp/local-volume-test-d1912c93-c852-4f94-8ac0-95cab3726d75] Namespace:persistent-local-volumes-test-2962 PodName:hostexec-node1-dfp4d ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:46:42.877: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "persistent-local-volumes-test-2962". STEP: Found 13 events. May 15 01:46:43.009: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for hostexec-node1-dfp4d: { } Scheduled: Successfully assigned persistent-local-volumes-test-2962/hostexec-node1-dfp4d to node1 May 15 01:46:43.009: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-e1744dfa-1547-4ca5-acdf-8eb7113868b1: { } Scheduled: Successfully assigned persistent-local-volumes-test-2962/pod-e1744dfa-1547-4ca5-acdf-8eb7113868b1 to node1 May 15 01:46:43.009: INFO: At 2021-05-15 01:44:34 +0000 UTC - event for hostexec-node1-dfp4d: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 15 01:46:43.009: INFO: At 2021-05-15 01:44:34 +0000 UTC - event for hostexec-node1-dfp4d: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 497.513423ms May 15 01:46:43.009: INFO: At 2021-05-15 01:44:34 +0000 UTC - event for hostexec-node1-dfp4d: {kubelet node1} Created: Created container agnhost-container May 15 01:46:43.009: INFO: At 2021-05-15 01:44:34 +0000 UTC - event for hostexec-node1-dfp4d: {kubelet node1} Started: Started container agnhost-container May 15 01:46:43.009: INFO: At 2021-05-15 01:44:45 +0000 UTC - event for pod-e1744dfa-1547-4ca5-acdf-8eb7113868b1: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:46:43.009: INFO: At 2021-05-15 01:44:45 +0000 UTC - event for pod-e1744dfa-1547-4ca5-acdf-8eb7113868b1: {multus } AddedInterface: Add eth0 [10.244.3.137/24] May 15 01:46:43.009: INFO: At 2021-05-15 01:44:46 +0000 UTC - event for pod-e1744dfa-1547-4ca5-acdf-8eb7113868b1: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:46:43.009: INFO: At 2021-05-15 01:44:46 +0000 UTC - event for pod-e1744dfa-1547-4ca5-acdf-8eb7113868b1: {kubelet node1} Failed: Error: ErrImagePull May 15 01:46:43.009: INFO: At 2021-05-15 01:44:47 +0000 UTC - event for pod-e1744dfa-1547-4ca5-acdf-8eb7113868b1: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 15 01:46:43.009: INFO: At 2021-05-15 01:44:47 +0000 UTC - event for pod-e1744dfa-1547-4ca5-acdf-8eb7113868b1: {kubelet node1} Failed: Error: ImagePullBackOff May 15 01:46:43.009: INFO: At 2021-05-15 01:46:16 +0000 UTC - event for pod-e1744dfa-1547-4ca5-acdf-8eb7113868b1: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:46:43.011: INFO: POD NODE PHASE GRACE CONDITIONS May 15 01:46:43.011: INFO: hostexec-node1-dfp4d node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:32 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:35 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:35 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:32 +0000 UTC }] May 15 01:46:43.012: INFO: pod-e1744dfa-1547-4ca5-acdf-8eb7113868b1 node1 Pending 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:42 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:42 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:42 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:44:42 +0000 UTC }] May 15 01:46:43.012: INFO: May 15 01:46:43.015: INFO: Logging node info for node master1 May 15 01:46:43.018: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 154660 0 2021-05-14 19:56:35 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:37 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:37 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:37 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:37 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[@ :],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:46:43.019: INFO: Logging kubelet events for node master1 May 15 01:46:43.024: INFO: Logging pods the kubelet thinks is on node master1 May 15 01:46:43.044: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 15 01:46:43.044: INFO: Container docker-registry ready: true, restart count 0 May 15 01:46:43.044: INFO: Container nginx ready: true, restart count 0 May 15 01:46:43.044: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.044: INFO: Container nfd-controller ready: true, restart count 0 May 15 01:46:43.044: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.044: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:46:43.044: INFO: kubernetes-metrics-scraper-678c97765c-fswrn started at 2021-05-15 00:18:49 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.044: INFO: Container kubernetes-metrics-scraper ready: true, restart count 0 May 15 01:46:43.044: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.044: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:46:43.044: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.044: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:43.044: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.044: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:46:43.044: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:43.044: INFO: Init container install-cni ready: true, restart count 0 May 15 01:46:43.044: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:46:43.044: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:46:43.044: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:43.044: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:43.044: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.044: INFO: Container kube-scheduler ready: true, restart count 0 W0515 01:46:43.057476 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:43.084: INFO: Latency metrics for node master1 May 15 01:46:43.084: INFO: Logging node info for node master2 May 15 01:46:43.087: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 154594 0 2021-05-14 19:57:04 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:35 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:35 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:35 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:35 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:46:43.087: INFO: Logging kubelet events for node master2 May 15 01:46:43.091: INFO: Logging pods the kubelet thinks is on node master2 May 15 01:46:43.100: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.100: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:46:43.100: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.100: INFO: Container kube-scheduler ready: true, restart count 2 May 15 01:46:43.100: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:43.100: INFO: Init container install-cni ready: true, restart count 0 May 15 01:46:43.100: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:46:43.100: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.100: INFO: Container autoscaler ready: true, restart count 2 May 15 01:46:43.100: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:46:43.100: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:43.100: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:43.100: INFO: prometheus-operator-5bb8cb9d8f-fqb87 started at 2021-05-15 00:18:49 +0000 UTC (0+2 container statuses recorded) May 15 01:46:43.100: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:43.100: INFO: Container prometheus-operator ready: true, restart count 0 May 15 01:46:43.100: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.100: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:46:43.100: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.100: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:43.100: INFO: coredns-7677f9bb54-96w24 started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.100: INFO: Container coredns ready: true, restart count 0 May 15 01:46:43.100: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.100: INFO: Container kube-proxy ready: true, restart count 2 W0515 01:46:43.112553 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:43.138: INFO: Latency metrics for node master2 May 15 01:46:43.138: INFO: Logging node info for node master3 May 15 01:46:43.140: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 154498 0 2021-05-14 19:57:14 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234767872 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324603904 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:33 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:33 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:33 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:33 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:46:43.140: INFO: Logging kubelet events for node master3 May 15 01:46:43.146: INFO: Logging pods the kubelet thinks is on node master3 May 15 01:46:43.155: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:43.155: INFO: Init container install-cni ready: true, restart count 0 May 15 01:46:43.156: INFO: Container kube-flannel ready: true, restart count 2 May 15 01:46:43.156: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.156: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:43.156: INFO: coredns-7677f9bb54-rpj8c started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.156: INFO: Container coredns ready: true, restart count 0 May 15 01:46:43.156: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:46:43.156: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:43.156: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:43.156: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.156: INFO: Container kube-controller-manager ready: true, restart count 3 May 15 01:46:43.156: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.156: INFO: Container kube-scheduler ready: true, restart count 3 May 15 01:46:43.156: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.156: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:46:43.156: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.156: INFO: Container kube-proxy ready: true, restart count 1 W0515 01:46:43.168713 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:43.197: INFO: Latency metrics for node master3 May 15 01:46:43.197: INFO: Logging node info for node node1 May 15 01:46:43.201: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 154736 0 2021-05-14 19:58:21 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-7734":"csi-mock-csi-mock-volumes-7734","csi-mock-csi-mock-volumes-9809":"csi-mock-csi-mock-volumes-9809"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:00 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:44:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:46:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:33 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:33 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:33 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:33 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[@ :],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:46:43.201: INFO: Logging kubelet events for node node1 May 15 01:46:43.206: INFO: Logging pods the kubelet thinks is on node node1 May 15 01:46:43.228: INFO: hostexec-node1-lmdw4 started at 2021-05-15 01:46:14 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.228: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:43.228: INFO: pod-604ae13e-a2a4-44cd-87c9-c2aa1f39edc0 started at 2021-05-15 01:46:28 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.228: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:43.228: INFO: csi-mockplugin-attacher-0 started at 2021-05-15 01:46:31 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.228: INFO: Container csi-attacher ready: true, restart count 0 May 15 01:46:43.228: INFO: csi-mockplugin-0 started at 2021-05-15 01:46:33 +0000 UTC (0+3 container statuses recorded) May 15 01:46:43.228: INFO: Container csi-provisioner ready: false, restart count 0 May 15 01:46:43.228: INFO: Container driver-registrar ready: false, restart count 0 May 15 01:46:43.228: INFO: Container mock ready: false, restart count 0 May 15 01:46:43.228: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:43.228: INFO: Init container install-cni ready: true, restart count 0 May 15 01:46:43.228: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:46:43.228: INFO: node-feature-discovery-worker-bw8zg started at 2021-05-15 00:18:56 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.228: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:46:43.228: INFO: collectd-mrzps started at 2021-05-15 00:19:22 +0000 UTC (0+3 container statuses recorded) May 15 01:46:43.228: INFO: Container collectd ready: true, restart count 0 May 15 01:46:43.228: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:46:43.228: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:46:43.228: INFO: hostexec-node1-5fw6h started at 2021-05-15 01:46:22 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.228: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:43.228: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.228: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:43.228: INFO: prometheus-k8s-0 started at 2021-05-15 00:19:01 +0000 UTC (0+5 container statuses recorded) May 15 01:46:43.228: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 15 01:46:43.228: INFO: Container grafana ready: true, restart count 0 May 15 01:46:43.228: INFO: Container prometheus ready: true, restart count 26 May 15 01:46:43.228: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 15 01:46:43.228: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 15 01:46:43.228: INFO: csi-mockplugin-resizer-0 started at 2021-05-15 01:46:33 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.228: INFO: Container csi-resizer ready: true, restart count 0 May 15 01:46:43.228: INFO: node-exporter-flvqz started at 2021-05-15 00:18:55 +0000 UTC (0+2 container statuses recorded) May 15 01:46:43.228: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:43.228: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:43.228: INFO: pod-c61f2051-9957-419e-995d-c3ed08d36192 started at 2021-05-15 01:46:20 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.228: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:43.228: INFO: hostexec-node1-r82s6 started at 2021-05-15 01:46:27 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.228: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:43.228: INFO: cmk-4s6dm started at 2021-05-15 00:18:54 +0000 UTC (0+2 container statuses recorded) May 15 01:46:43.228: INFO: Container nodereport ready: true, restart count 0 May 15 01:46:43.228: INFO: Container reconcile ready: true, restart count 0 May 15 01:46:43.228: INFO: hostexec-node1-dfp4d started at 2021-05-15 01:44:32 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.228: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:43.228: INFO: pod-dbfe8699-c076-46f1-bf35-86df11b4addd started at (0+0 container statuses recorded) May 15 01:46:43.228: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.228: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:46:43.228: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4vxcc started at 2021-05-15 00:19:00 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.228: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:46:43.228: INFO: hostexec-node1-frmv7 started at 2021-05-15 01:46:31 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.228: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:46:43.228: INFO: pod-e1744dfa-1547-4ca5-acdf-8eb7113868b1 started at 2021-05-15 01:44:42 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.228: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:43.228: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.228: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:46:43.229: INFO: pod-4cdc198f-2eab-46aa-b552-61f5a2bf0de0 started at 2021-05-15 01:46:33 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.229: INFO: Container write-pod ready: false, restart count 0 May 15 01:46:43.229: INFO: csi-mockplugin-0 started at 2021-05-15 01:46:31 +0000 UTC (0+3 container statuses recorded) May 15 01:46:43.229: INFO: Container csi-provisioner ready: false, restart count 0 May 15 01:46:43.229: INFO: Container driver-registrar ready: false, restart count 0 May 15 01:46:43.229: INFO: Container mock ready: false, restart count 0 May 15 01:46:43.229: INFO: hostexec-node1-sv6vb started at 2021-05-15 01:46:26 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.229: INFO: Container agnhost-container ready: true, restart count 0 W0515 01:46:43.241059 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:43.274: INFO: Latency metrics for node node1 May 15 01:46:43.274: INFO: Logging node info for node node2 May 15 01:46:43.277: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 154695 0 2021-05-14 19:58:22 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-3052":"csi-mock-csi-mock-volumes-3052","csi-mock-csi-mock-volumes-8760":"csi-mock-csi-mock-volumes-8760"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:46 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:44:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}},"f:status":{"f:volumesAttached":{}}}} {kubelet Update v1 2021-05-15 01:44:49 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}},"f:volumesInUse":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:39 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:39 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:46:39 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:46:39 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-3052^4],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-3052^4,DevicePath:,},},Config:nil,},} May 15 01:46:43.278: INFO: Logging kubelet events for node node2 May 15 01:46:43.282: INFO: Logging pods the kubelet thinks is on node node2 May 15 01:46:43.295: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.295: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:46:43.295: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.295: INFO: Container kube-multus ready: true, restart count 1 May 15 01:46:43.295: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.295: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 15 01:46:43.295: INFO: pvc-volume-tester-8nftk started at 2021-05-15 01:44:47 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.295: INFO: Container volume-tester ready: true, restart count 0 May 15 01:46:43.295: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.295: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:46:43.295: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:46:43.295: INFO: Init container install-cni ready: true, restart count 1 May 15 01:46:43.295: INFO: Container kube-flannel ready: true, restart count 4 May 15 01:46:43.295: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:46:43.295: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:46:43.295: INFO: Container node-exporter ready: true, restart count 0 May 15 01:46:43.295: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 15 01:46:43.295: INFO: Container tas-controller ready: true, restart count 0 May 15 01:46:43.295: INFO: Container tas-extender ready: true, restart count 0 May 15 01:46:43.295: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 15 01:46:43.295: INFO: Container collectd ready: true, restart count 0 May 15 01:46:43.295: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:46:43.295: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:46:43.295: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.295: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:46:43.295: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.295: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:46:43.295: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 15 01:46:43.295: INFO: Container nodereport ready: true, restart count 0 May 15 01:46:43.295: INFO: Container reconcile ready: true, restart count 0 May 15 01:46:43.295: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.295: INFO: Container cmk-webhook ready: true, restart count 0 May 15 01:46:43.295: INFO: csi-mockplugin-0 started at 2021-05-15 01:44:36 +0000 UTC (0+3 container statuses recorded) May 15 01:46:43.295: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:46:43.295: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:46:43.295: INFO: Container mock ready: true, restart count 0 May 15 01:46:43.295: INFO: csi-mockplugin-attacher-0 started at 2021-05-15 01:44:36 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.295: INFO: Container csi-attacher ready: true, restart count 0 May 15 01:46:43.295: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 15 01:46:43.295: INFO: Container discover ready: false, restart count 0 May 15 01:46:43.295: INFO: Container init ready: false, restart count 0 May 15 01:46:43.295: INFO: Container install ready: false, restart count 0 May 15 01:46:43.295: INFO: csi-mockplugin-0 started at 2021-05-15 01:46:19 +0000 UTC (0+3 container statuses recorded) May 15 01:46:43.295: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:46:43.295: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:46:43.295: INFO: Container mock ready: true, restart count 0 May 15 01:46:43.295: INFO: csi-mockplugin-attacher-0 started at 2021-05-15 01:46:19 +0000 UTC (0+1 container statuses recorded) May 15 01:46:43.295: INFO: Container csi-attacher ready: true, restart count 0 W0515 01:46:43.310984 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:46:43.352: INFO: Latency metrics for node node2 May 15 01:46:43.352: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-2962" for this suite. • Failure in Spec Setup (BeforeEach) [130.917 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: dir-bindmounted] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:205 should be able to mount volume and write from pod1 [BeforeEach] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:234 May 15 01:46:42.861: Unexpected error: <*errors.errorString | 0xc002c6c350>: { s: "pod \"pod-e1744dfa-1547-4ca5-acdf-8eb7113868b1\" is not Running: timed out waiting for the condition", } pod "pod-e1744dfa-1547-4ca5-acdf-8eb7113868b1" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 ------------------------------ {"msg":"FAILED [sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] One pod requesting one prebound PVC should be able to mount volume and write from pod1","total":-1,"completed":1,"skipped":93,"failed":1,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] One pod requesting one prebound PVC should be able to mount volume and write from pod1"]} SSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:46:19.003: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename csi-mock-volumes STEP: Waiting for a default service account to be provisioned in namespace [It] contain ephemeral=true when using inline volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:358 STEP: Building a driver namespace object, basename csi-mock-volumes-8760 STEP: Waiting for a default service account to be provisioned in namespace STEP: deploying csi mock driver May 15 01:46:19.069: INFO: creating *v1.ServiceAccount: csi-mock-volumes-8760-7340/csi-attacher May 15 01:46:19.072: INFO: creating *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-8760 May 15 01:46:19.072: INFO: Define cluster role external-attacher-runner-csi-mock-volumes-8760 May 15 01:46:19.074: INFO: creating *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-8760 May 15 01:46:19.078: INFO: creating *v1.Role: csi-mock-volumes-8760-7340/external-attacher-cfg-csi-mock-volumes-8760 May 15 01:46:19.084: INFO: creating *v1.RoleBinding: csi-mock-volumes-8760-7340/csi-attacher-role-cfg May 15 01:46:19.090: INFO: creating *v1.ServiceAccount: csi-mock-volumes-8760-7340/csi-provisioner May 15 01:46:19.096: INFO: creating *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-8760 May 15 01:46:19.096: INFO: Define cluster role external-provisioner-runner-csi-mock-volumes-8760 May 15 01:46:19.101: INFO: creating *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-8760 May 15 01:46:19.105: INFO: creating *v1.Role: csi-mock-volumes-8760-7340/external-provisioner-cfg-csi-mock-volumes-8760 May 15 01:46:19.107: INFO: creating *v1.RoleBinding: csi-mock-volumes-8760-7340/csi-provisioner-role-cfg May 15 01:46:19.110: INFO: creating *v1.ServiceAccount: csi-mock-volumes-8760-7340/csi-resizer May 15 01:46:19.113: INFO: creating *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-8760 May 15 01:46:19.113: INFO: Define cluster role external-resizer-runner-csi-mock-volumes-8760 May 15 01:46:19.116: INFO: creating *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-8760 May 15 01:46:19.118: INFO: creating *v1.Role: csi-mock-volumes-8760-7340/external-resizer-cfg-csi-mock-volumes-8760 May 15 01:46:19.121: INFO: creating *v1.RoleBinding: csi-mock-volumes-8760-7340/csi-resizer-role-cfg May 15 01:46:19.123: INFO: creating *v1.ServiceAccount: csi-mock-volumes-8760-7340/csi-mock May 15 01:46:19.126: INFO: creating *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-8760 May 15 01:46:19.129: INFO: creating *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-8760 May 15 01:46:19.131: INFO: creating *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-8760 May 15 01:46:19.133: INFO: creating *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-8760 May 15 01:46:19.136: INFO: creating *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-8760 May 15 01:46:19.138: INFO: creating *v1.StorageClass: csi-mock-sc-csi-mock-volumes-8760 May 15 01:46:19.140: INFO: creating *v1.StatefulSet: csi-mock-volumes-8760-7340/csi-mockplugin May 15 01:46:19.146: INFO: creating *v1.CSIDriver: csi-mock-csi-mock-volumes-8760 May 15 01:46:19.149: INFO: creating *v1.StatefulSet: csi-mock-volumes-8760-7340/csi-mockplugin-attacher May 15 01:46:19.152: INFO: waiting up to 4m0s for CSIDriver "csi-mock-csi-mock-volumes-8760" May 15 01:46:19.154: INFO: waiting for CSIDriver csi-mock-csi-mock-volumes-8760 to register on node node2 STEP: Creating pod STEP: checking for CSIInlineVolumes feature May 15 01:46:28.192: INFO: Pod inline-volume-mq7f5 has the following logs: May 15 01:46:28.192: INFO: Deleting pod "inline-volume-mq7f5" in namespace "csi-mock-volumes-8760" May 15 01:46:28.196: INFO: Wait up to 5m0s for pod "inline-volume-mq7f5" to be fully deleted STEP: Deleting the previously created pod May 15 01:46:30.201: INFO: Deleting pod "pvc-volume-tester-jq8ps" in namespace "csi-mock-volumes-8760" May 15 01:46:30.207: INFO: Wait up to 5m0s for pod "pvc-volume-tester-jq8ps" to be fully deleted STEP: Checking CSI driver logs May 15 01:46:40.223: INFO: Found volume attribute csi.storage.k8s.io/pod.name: pvc-volume-tester-jq8ps May 15 01:46:40.223: INFO: Found volume attribute csi.storage.k8s.io/pod.namespace: csi-mock-volumes-8760 May 15 01:46:40.223: INFO: Found volume attribute csi.storage.k8s.io/pod.uid: 695660b4-9955-4252-b575-85934722057a May 15 01:46:40.223: INFO: Found volume attribute csi.storage.k8s.io/ephemeral: true May 15 01:46:40.223: INFO: Found volume attribute csi.storage.k8s.io/serviceAccount.name: default May 15 01:46:40.223: INFO: Found NodeUnpublishVolume: {json: {"Method":"/csi.v1.Node/NodeUnpublishVolume","Request":{"volume_id":"csi-8f2ed98c1ec435f7c4a80729891288b134cf4341a0debad0c44131272fc8c354","target_path":"/var/lib/kubelet/pods/695660b4-9955-4252-b575-85934722057a/volumes/kubernetes.io~csi/my-volume/mount"},"Response":{},"Error":"","FullError":null} Method:NodeUnpublishVolume Request:{VolumeContext:map[]} FullError:{Code:OK Message:} Error:} STEP: Deleting pod pvc-volume-tester-jq8ps May 15 01:46:40.223: INFO: Deleting pod "pvc-volume-tester-jq8ps" in namespace "csi-mock-volumes-8760" STEP: Cleaning up resources STEP: deleting the test namespace: csi-mock-volumes-8760 STEP: Waiting for namespaces [csi-mock-volumes-8760] to vanish STEP: uninstalling csi mock driver May 15 01:46:46.241: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-8760-7340/csi-attacher May 15 01:46:46.246: INFO: deleting *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-8760 May 15 01:46:46.251: INFO: deleting *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-8760 May 15 01:46:46.255: INFO: deleting *v1.Role: csi-mock-volumes-8760-7340/external-attacher-cfg-csi-mock-volumes-8760 May 15 01:46:46.259: INFO: deleting *v1.RoleBinding: csi-mock-volumes-8760-7340/csi-attacher-role-cfg May 15 01:46:46.263: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-8760-7340/csi-provisioner May 15 01:46:46.267: INFO: deleting *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-8760 May 15 01:46:46.271: INFO: deleting *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-8760 May 15 01:46:46.276: INFO: deleting *v1.Role: csi-mock-volumes-8760-7340/external-provisioner-cfg-csi-mock-volumes-8760 May 15 01:46:46.281: INFO: deleting *v1.RoleBinding: csi-mock-volumes-8760-7340/csi-provisioner-role-cfg May 15 01:46:46.286: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-8760-7340/csi-resizer May 15 01:46:46.290: INFO: deleting *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-8760 May 15 01:46:46.295: INFO: deleting *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-8760 May 15 01:46:46.299: INFO: deleting *v1.Role: csi-mock-volumes-8760-7340/external-resizer-cfg-csi-mock-volumes-8760 May 15 01:46:46.303: INFO: deleting *v1.RoleBinding: csi-mock-volumes-8760-7340/csi-resizer-role-cfg May 15 01:46:46.307: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-8760-7340/csi-mock May 15 01:46:46.311: INFO: deleting *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-8760 May 15 01:46:46.315: INFO: deleting *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-8760 May 15 01:46:46.319: INFO: deleting *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-8760 May 15 01:46:46.323: INFO: deleting *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-8760 May 15 01:46:46.327: INFO: deleting *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-8760 May 15 01:46:46.331: INFO: deleting *v1.StorageClass: csi-mock-sc-csi-mock-volumes-8760 May 15 01:46:46.335: INFO: deleting *v1.StatefulSet: csi-mock-volumes-8760-7340/csi-mockplugin May 15 01:46:46.340: INFO: deleting *v1.CSIDriver: csi-mock-csi-mock-volumes-8760 May 15 01:46:46.344: INFO: deleting *v1.StatefulSet: csi-mock-volumes-8760-7340/csi-mockplugin-attacher STEP: deleting the driver namespace: csi-mock-volumes-8760-7340 STEP: Waiting for namespaces [csi-mock-volumes-8760-7340] to vanish [AfterEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:46:52.359: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "csi-mock-volumes-8760" for this suite. May 15 01:46:52.365: INFO: Namespace csi-mock-volumes-8760 was already deleted STEP: Destroying namespace "csi-mock-volumes-8760-7340" for this suite. May 15 01:46:52.368: INFO: Namespace csi-mock-volumes-8760-7340 was already deleted • [SLOW TEST:33.366 seconds] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 CSI workload information using mock driver /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:308 contain ephemeral=true when using inline volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:358 ------------------------------ {"msg":"PASSED [sig-storage] CSI mock volume CSI workload information using mock driver contain ephemeral=true when using inline volume","total":-1,"completed":1,"skipped":353,"failed":1,"failures":["[sig-storage] PersistentVolumes-local [Volume type: tmpfs] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2"]} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:46:43.404: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename csi-mock-volumes STEP: Waiting for a default service account to be provisioned in namespace [It] exhausted, late binding, with topology /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:805 STEP: Building a driver namespace object, basename csi-mock-volumes-9624 STEP: Waiting for a default service account to be provisioned in namespace STEP: deploying csi mock driver May 15 01:46:43.470: INFO: creating *v1.ServiceAccount: csi-mock-volumes-9624-1365/csi-attacher May 15 01:46:43.473: INFO: creating *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-9624 May 15 01:46:43.473: INFO: Define cluster role external-attacher-runner-csi-mock-volumes-9624 May 15 01:46:43.476: INFO: creating *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-9624 May 15 01:46:43.479: INFO: creating *v1.Role: csi-mock-volumes-9624-1365/external-attacher-cfg-csi-mock-volumes-9624 May 15 01:46:43.482: INFO: creating *v1.RoleBinding: csi-mock-volumes-9624-1365/csi-attacher-role-cfg May 15 01:46:43.484: INFO: creating *v1.ServiceAccount: csi-mock-volumes-9624-1365/csi-provisioner May 15 01:46:43.486: INFO: creating *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-9624 May 15 01:46:43.486: INFO: Define cluster role external-provisioner-runner-csi-mock-volumes-9624 May 15 01:46:43.489: INFO: creating *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-9624 May 15 01:46:43.492: INFO: creating *v1.Role: csi-mock-volumes-9624-1365/external-provisioner-cfg-csi-mock-volumes-9624 May 15 01:46:43.495: INFO: creating *v1.RoleBinding: csi-mock-volumes-9624-1365/csi-provisioner-role-cfg May 15 01:46:43.497: INFO: creating *v1.ServiceAccount: csi-mock-volumes-9624-1365/csi-resizer May 15 01:46:43.500: INFO: creating *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-9624 May 15 01:46:43.500: INFO: Define cluster role external-resizer-runner-csi-mock-volumes-9624 May 15 01:46:43.502: INFO: creating *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-9624 May 15 01:46:43.505: INFO: creating *v1.Role: csi-mock-volumes-9624-1365/external-resizer-cfg-csi-mock-volumes-9624 May 15 01:46:43.507: INFO: creating *v1.RoleBinding: csi-mock-volumes-9624-1365/csi-resizer-role-cfg May 15 01:46:43.509: INFO: creating *v1.ServiceAccount: csi-mock-volumes-9624-1365/csi-mock May 15 01:46:43.512: INFO: creating *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-9624 May 15 01:46:43.514: INFO: creating *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-9624 May 15 01:46:43.516: INFO: creating *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-9624 May 15 01:46:43.519: INFO: creating *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-9624 May 15 01:46:43.521: INFO: creating *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-9624 May 15 01:46:43.524: INFO: creating *v1.StorageClass: csi-mock-sc-csi-mock-volumes-9624 May 15 01:46:43.527: INFO: creating *v1.StatefulSet: csi-mock-volumes-9624-1365/csi-mockplugin May 15 01:46:43.532: INFO: creating *v1.CSIDriver: csi-mock-csi-mock-volumes-9624 May 15 01:46:43.534: INFO: waiting up to 4m0s for CSIDriver "csi-mock-csi-mock-volumes-9624" May 15 01:46:43.536: INFO: waiting for CSIDriver csi-mock-csi-mock-volumes-9624 to register on node node2 STEP: Creating pod May 15 01:46:53.055: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil May 15 01:46:59.076: INFO: Deleting pod "pvc-volume-tester-grxkn" in namespace "csi-mock-volumes-9624" May 15 01:46:59.081: INFO: Wait up to 5m0s for pod "pvc-volume-tester-grxkn" to be fully deleted STEP: Checking PVC events May 15 01:47:04.109: INFO: PVC event ADDED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-ps4mf", GenerateName:"pvc-", Namespace:"csi-mock-volumes-9624", SelfLink:"/api/v1/namespaces/csi-mock-volumes-9624/persistentvolumeclaims/pvc-ps4mf", UID:"060df178-384e-4345-aee5-05679b05dda4", ResourceVersion:"155051", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63756640013, loc:(*time.Location)(0x770c940)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"e2e.test", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc0018270c0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0018270e0)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"", StorageClassName:(*string)(0xc004dfb3e0), VolumeMode:(*v1.PersistentVolumeMode)(0xc004dfb3f0), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Pending", AccessModes:[]v1.PersistentVolumeAccessMode(nil), Capacity:v1.ResourceList(nil), Conditions:[]v1.PersistentVolumeClaimCondition(nil)}} May 15 01:47:04.109: INFO: PVC event MODIFIED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-ps4mf", GenerateName:"pvc-", Namespace:"csi-mock-volumes-9624", SelfLink:"/api/v1/namespaces/csi-mock-volumes-9624/persistentvolumeclaims/pvc-ps4mf", UID:"060df178-384e-4345-aee5-05679b05dda4", ResourceVersion:"155054", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63756640013, loc:(*time.Location)(0x770c940)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string{"volume.kubernetes.io/selected-node":"node2"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"e2e.test", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc003933180), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0039331c0)}, v1.ManagedFieldsEntry{Manager:"kube-scheduler", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc003933220), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc003933260)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"", StorageClassName:(*string)(0xc000f832f0), VolumeMode:(*v1.PersistentVolumeMode)(0xc000f83330), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Pending", AccessModes:[]v1.PersistentVolumeAccessMode(nil), Capacity:v1.ResourceList(nil), Conditions:[]v1.PersistentVolumeClaimCondition(nil)}} May 15 01:47:04.109: INFO: PVC event MODIFIED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-ps4mf", GenerateName:"pvc-", Namespace:"csi-mock-volumes-9624", SelfLink:"/api/v1/namespaces/csi-mock-volumes-9624/persistentvolumeclaims/pvc-ps4mf", UID:"060df178-384e-4345-aee5-05679b05dda4", ResourceVersion:"155055", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63756640013, loc:(*time.Location)(0x770c940)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string{"volume.beta.kubernetes.io/storage-provisioner":"csi-mock-csi-mock-volumes-9624", "volume.kubernetes.io/selected-node":"node2"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"e2e.test", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc001827e20), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc001827e40)}, v1.ManagedFieldsEntry{Manager:"kube-controller-manager", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc001827e60), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc001827e80)}, v1.ManagedFieldsEntry{Manager:"kube-scheduler", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc001827ea0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc001827ec0)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"", StorageClassName:(*string)(0xc004dfb9b0), VolumeMode:(*v1.PersistentVolumeMode)(0xc004dfb9c0), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Pending", AccessModes:[]v1.PersistentVolumeAccessMode(nil), Capacity:v1.ResourceList(nil), Conditions:[]v1.PersistentVolumeClaimCondition(nil)}} May 15 01:47:04.110: INFO: PVC event MODIFIED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-ps4mf", GenerateName:"pvc-", Namespace:"csi-mock-volumes-9624", SelfLink:"/api/v1/namespaces/csi-mock-volumes-9624/persistentvolumeclaims/pvc-ps4mf", UID:"060df178-384e-4345-aee5-05679b05dda4", ResourceVersion:"155059", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63756640013, loc:(*time.Location)(0x770c940)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string{"volume.beta.kubernetes.io/storage-provisioner":"csi-mock-csi-mock-volumes-9624"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"e2e.test", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc001995040), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc001995060)}, v1.ManagedFieldsEntry{Manager:"kube-controller-manager", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc001995080), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0019950a0)}, v1.ManagedFieldsEntry{Manager:"kube-scheduler", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc0019950c0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0019950e0)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"", StorageClassName:(*string)(0xc004e18890), VolumeMode:(*v1.PersistentVolumeMode)(0xc004e188a0), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Pending", AccessModes:[]v1.PersistentVolumeAccessMode(nil), Capacity:v1.ResourceList(nil), Conditions:[]v1.PersistentVolumeClaimCondition(nil)}} May 15 01:47:04.110: INFO: PVC event MODIFIED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-ps4mf", GenerateName:"pvc-", Namespace:"csi-mock-volumes-9624", SelfLink:"/api/v1/namespaces/csi-mock-volumes-9624/persistentvolumeclaims/pvc-ps4mf", UID:"060df178-384e-4345-aee5-05679b05dda4", ResourceVersion:"155073", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63756640013, loc:(*time.Location)(0x770c940)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string{"volume.beta.kubernetes.io/storage-provisioner":"csi-mock-csi-mock-volumes-9624", "volume.kubernetes.io/selected-node":"node2"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"e2e.test", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc001995120), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc001995140)}, v1.ManagedFieldsEntry{Manager:"kube-controller-manager", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc001995160), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc001995180)}, v1.ManagedFieldsEntry{Manager:"kube-scheduler", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc0019951a0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0019951c0)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"", StorageClassName:(*string)(0xc004e188d0), VolumeMode:(*v1.PersistentVolumeMode)(0xc004e188e0), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Pending", AccessModes:[]v1.PersistentVolumeAccessMode(nil), Capacity:v1.ResourceList(nil), Conditions:[]v1.PersistentVolumeClaimCondition(nil)}} May 15 01:47:04.110: INFO: PVC event MODIFIED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-ps4mf", GenerateName:"pvc-", Namespace:"csi-mock-volumes-9624", SelfLink:"/api/v1/namespaces/csi-mock-volumes-9624/persistentvolumeclaims/pvc-ps4mf", UID:"060df178-384e-4345-aee5-05679b05dda4", ResourceVersion:"155079", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63756640013, loc:(*time.Location)(0x770c940)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string{"pv.kubernetes.io/bind-completed":"yes", "pv.kubernetes.io/bound-by-controller":"yes", "volume.beta.kubernetes.io/storage-provisioner":"csi-mock-csi-mock-volumes-9624", "volume.kubernetes.io/selected-node":"node2"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"e2e.test", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc001b12280), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc001b122a0)}, v1.ManagedFieldsEntry{Manager:"kube-controller-manager", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc001b122c0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc001b122e0)}, v1.ManagedFieldsEntry{Manager:"kube-scheduler", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc001b12300), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc001b12320)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"pvc-060df178-384e-4345-aee5-05679b05dda4", StorageClassName:(*string)(0xc004dfbe50), VolumeMode:(*v1.PersistentVolumeMode)(0xc004dfbe60), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Pending", AccessModes:[]v1.PersistentVolumeAccessMode(nil), Capacity:v1.ResourceList(nil), Conditions:[]v1.PersistentVolumeClaimCondition(nil)}} May 15 01:47:04.110: INFO: PVC event MODIFIED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-ps4mf", GenerateName:"pvc-", Namespace:"csi-mock-volumes-9624", SelfLink:"/api/v1/namespaces/csi-mock-volumes-9624/persistentvolumeclaims/pvc-ps4mf", UID:"060df178-384e-4345-aee5-05679b05dda4", ResourceVersion:"155080", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63756640013, loc:(*time.Location)(0x770c940)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string{"pv.kubernetes.io/bind-completed":"yes", "pv.kubernetes.io/bound-by-controller":"yes", "volume.beta.kubernetes.io/storage-provisioner":"csi-mock-csi-mock-volumes-9624", "volume.kubernetes.io/selected-node":"node2"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"e2e.test", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc001b12360), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc001b12380)}, v1.ManagedFieldsEntry{Manager:"kube-controller-manager", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc001b123a0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc001b123c0)}, v1.ManagedFieldsEntry{Manager:"kube-scheduler", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc001b123e0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc001b12400)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"pvc-060df178-384e-4345-aee5-05679b05dda4", StorageClassName:(*string)(0xc004dfbe90), VolumeMode:(*v1.PersistentVolumeMode)(0xc004dfbea0), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Bound", AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Capacity:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}, Conditions:[]v1.PersistentVolumeClaimCondition(nil)}} May 15 01:47:04.110: INFO: PVC event MODIFIED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-ps4mf", GenerateName:"pvc-", Namespace:"csi-mock-volumes-9624", SelfLink:"/api/v1/namespaces/csi-mock-volumes-9624/persistentvolumeclaims/pvc-ps4mf", UID:"060df178-384e-4345-aee5-05679b05dda4", ResourceVersion:"155188", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63756640013, loc:(*time.Location)(0x770c940)}}, DeletionTimestamp:(*v1.Time)(0xc001995200), DeletionGracePeriodSeconds:(*int64)(0xc0036b3ba8), Labels:map[string]string(nil), Annotations:map[string]string{"pv.kubernetes.io/bind-completed":"yes", "pv.kubernetes.io/bound-by-controller":"yes", "volume.beta.kubernetes.io/storage-provisioner":"csi-mock-csi-mock-volumes-9624", "volume.kubernetes.io/selected-node":"node2"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"e2e.test", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc001995220), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc001995240)}, v1.ManagedFieldsEntry{Manager:"kube-controller-manager", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc001995260), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc001995280)}, v1.ManagedFieldsEntry{Manager:"kube-scheduler", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc0019952a0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0019952c0)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"pvc-060df178-384e-4345-aee5-05679b05dda4", StorageClassName:(*string)(0xc004e18910), VolumeMode:(*v1.PersistentVolumeMode)(0xc004e18920), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Bound", AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Capacity:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}, Conditions:[]v1.PersistentVolumeClaimCondition(nil)}} May 15 01:47:04.110: INFO: PVC event DELETED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-ps4mf", GenerateName:"pvc-", Namespace:"csi-mock-volumes-9624", SelfLink:"/api/v1/namespaces/csi-mock-volumes-9624/persistentvolumeclaims/pvc-ps4mf", UID:"060df178-384e-4345-aee5-05679b05dda4", ResourceVersion:"155189", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63756640013, loc:(*time.Location)(0x770c940)}}, DeletionTimestamp:(*v1.Time)(0xc001b12440), DeletionGracePeriodSeconds:(*int64)(0xc0040841b8), Labels:map[string]string(nil), Annotations:map[string]string{"pv.kubernetes.io/bind-completed":"yes", "pv.kubernetes.io/bound-by-controller":"yes", "volume.beta.kubernetes.io/storage-provisioner":"csi-mock-csi-mock-volumes-9624", "volume.kubernetes.io/selected-node":"node2"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"e2e.test", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc001b12460), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc001b12480)}, v1.ManagedFieldsEntry{Manager:"kube-controller-manager", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc001b124a0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc001b124c0)}, v1.ManagedFieldsEntry{Manager:"kube-scheduler", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc001b124e0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc001b12500)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"pvc-060df178-384e-4345-aee5-05679b05dda4", StorageClassName:(*string)(0xc004dfbee0), VolumeMode:(*v1.PersistentVolumeMode)(0xc004dfbef0), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Bound", AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Capacity:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}, Conditions:[]v1.PersistentVolumeClaimCondition(nil)}} STEP: Deleting pod pvc-volume-tester-grxkn May 15 01:47:04.110: INFO: Deleting pod "pvc-volume-tester-grxkn" in namespace "csi-mock-volumes-9624" STEP: Deleting claim pvc-ps4mf STEP: Deleting storageclass csi-mock-volumes-9624-sc STEP: Cleaning up resources STEP: deleting the test namespace: csi-mock-volumes-9624 STEP: Waiting for namespaces [csi-mock-volumes-9624] to vanish STEP: uninstalling csi mock driver May 15 01:47:10.133: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-9624-1365/csi-attacher May 15 01:47:10.138: INFO: deleting *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-9624 May 15 01:47:10.143: INFO: deleting *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-9624 May 15 01:47:10.147: INFO: deleting *v1.Role: csi-mock-volumes-9624-1365/external-attacher-cfg-csi-mock-volumes-9624 May 15 01:47:10.152: INFO: deleting *v1.RoleBinding: csi-mock-volumes-9624-1365/csi-attacher-role-cfg May 15 01:47:10.156: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-9624-1365/csi-provisioner May 15 01:47:10.160: INFO: deleting *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-9624 May 15 01:47:10.164: INFO: deleting *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-9624 May 15 01:47:10.168: INFO: deleting *v1.Role: csi-mock-volumes-9624-1365/external-provisioner-cfg-csi-mock-volumes-9624 May 15 01:47:10.172: INFO: deleting *v1.RoleBinding: csi-mock-volumes-9624-1365/csi-provisioner-role-cfg May 15 01:47:10.176: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-9624-1365/csi-resizer May 15 01:47:10.180: INFO: deleting *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-9624 May 15 01:47:10.184: INFO: deleting *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-9624 May 15 01:47:10.189: INFO: deleting *v1.Role: csi-mock-volumes-9624-1365/external-resizer-cfg-csi-mock-volumes-9624 May 15 01:47:10.193: INFO: deleting *v1.RoleBinding: csi-mock-volumes-9624-1365/csi-resizer-role-cfg May 15 01:47:10.197: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-9624-1365/csi-mock May 15 01:47:10.202: INFO: deleting *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-9624 May 15 01:47:10.206: INFO: deleting *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-9624 May 15 01:47:10.210: INFO: deleting *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-9624 May 15 01:47:10.215: INFO: deleting *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-9624 May 15 01:47:10.219: INFO: deleting *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-9624 May 15 01:47:10.224: INFO: deleting *v1.StorageClass: csi-mock-sc-csi-mock-volumes-9624 May 15 01:47:10.229: INFO: deleting *v1.StatefulSet: csi-mock-volumes-9624-1365/csi-mockplugin May 15 01:47:10.234: INFO: deleting *v1.CSIDriver: csi-mock-csi-mock-volumes-9624 STEP: deleting the driver namespace: csi-mock-volumes-9624-1365 STEP: Waiting for namespaces [csi-mock-volumes-9624-1365] to vanish [AfterEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:47:22.248: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "csi-mock-volumes-9624" for this suite. May 15 01:47:22.255: INFO: Namespace csi-mock-volumes-9624 was already deleted STEP: Destroying namespace "csi-mock-volumes-9624-1365" for this suite. May 15 01:47:22.257: INFO: Namespace csi-mock-volumes-9624-1365 was already deleted • [SLOW TEST:38.854 seconds] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 storage capacity /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:752 exhausted, late binding, with topology /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:805 ------------------------------ {"msg":"PASSED [sig-storage] CSI mock volume storage capacity exhausted, late binding, with topology","total":-1,"completed":2,"skipped":109,"failed":1,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] One pod requesting one prebound PVC should be able to mount volume and write from pod1"]} SSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes GCEPD /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:47:22.277: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename pv STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes GCEPD /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-gce.go:77 May 15 01:47:22.298: INFO: Only supported for providers [gce gke] (not skeleton) [AfterEach] [sig-storage] PersistentVolumes GCEPD /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:47:22.300: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "pv-4278" for this suite. [AfterEach] [sig-storage] PersistentVolumes GCEPD /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-gce.go:110 May 15 01:47:22.306: INFO: AfterEach: Cleaning up test resources May 15 01:47:22.306: INFO: pvc is nil May 15 01:47:22.306: INFO: pv is nil S [SKIPPING] in Spec Setup (BeforeEach) [0.029 seconds] [sig-storage] PersistentVolumes GCEPD /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 should test that deleting a PVC before the pod does not cause pod deletion to fail on PD detach [BeforeEach] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-gce.go:126 Only supported for providers [gce gke] (not skeleton) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-gce.go:85 ------------------------------ SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:46:33.637: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename csi-mock-volumes STEP: Waiting for a default service account to be provisioned in namespace [It] should expand volume by restarting pod if attach=off, nodeExpansion=on /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:455 STEP: Building a driver namespace object, basename csi-mock-volumes-4249 STEP: Waiting for a default service account to be provisioned in namespace STEP: deploying csi mock driver May 15 01:46:33.713: INFO: creating *v1.ServiceAccount: csi-mock-volumes-4249-4927/csi-attacher May 15 01:46:33.716: INFO: creating *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-4249 May 15 01:46:33.716: INFO: Define cluster role external-attacher-runner-csi-mock-volumes-4249 May 15 01:46:33.718: INFO: creating *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-4249 May 15 01:46:33.721: INFO: creating *v1.Role: csi-mock-volumes-4249-4927/external-attacher-cfg-csi-mock-volumes-4249 May 15 01:46:33.723: INFO: creating *v1.RoleBinding: csi-mock-volumes-4249-4927/csi-attacher-role-cfg May 15 01:46:33.725: INFO: creating *v1.ServiceAccount: csi-mock-volumes-4249-4927/csi-provisioner May 15 01:46:33.728: INFO: creating *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-4249 May 15 01:46:33.728: INFO: Define cluster role external-provisioner-runner-csi-mock-volumes-4249 May 15 01:46:33.731: INFO: creating *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-4249 May 15 01:46:33.734: INFO: creating *v1.Role: csi-mock-volumes-4249-4927/external-provisioner-cfg-csi-mock-volumes-4249 May 15 01:46:33.737: INFO: creating *v1.RoleBinding: csi-mock-volumes-4249-4927/csi-provisioner-role-cfg May 15 01:46:33.740: INFO: creating *v1.ServiceAccount: csi-mock-volumes-4249-4927/csi-resizer May 15 01:46:33.742: INFO: creating *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-4249 May 15 01:46:33.742: INFO: Define cluster role external-resizer-runner-csi-mock-volumes-4249 May 15 01:46:33.745: INFO: creating *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-4249 May 15 01:46:33.747: INFO: creating *v1.Role: csi-mock-volumes-4249-4927/external-resizer-cfg-csi-mock-volumes-4249 May 15 01:46:33.749: INFO: creating *v1.RoleBinding: csi-mock-volumes-4249-4927/csi-resizer-role-cfg May 15 01:46:33.753: INFO: creating *v1.ServiceAccount: csi-mock-volumes-4249-4927/csi-mock May 15 01:46:33.756: INFO: creating *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-4249 May 15 01:46:33.758: INFO: creating *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-4249 May 15 01:46:33.761: INFO: creating *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-4249 May 15 01:46:33.764: INFO: creating *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-4249 May 15 01:46:33.766: INFO: creating *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-4249 May 15 01:46:33.769: INFO: creating *v1.StorageClass: csi-mock-sc-csi-mock-volumes-4249 May 15 01:46:33.771: INFO: creating *v1.StatefulSet: csi-mock-volumes-4249-4927/csi-mockplugin May 15 01:46:33.776: INFO: creating *v1.CSIDriver: csi-mock-csi-mock-volumes-4249 May 15 01:46:33.779: INFO: creating *v1.StatefulSet: csi-mock-volumes-4249-4927/csi-mockplugin-resizer May 15 01:46:33.782: INFO: waiting up to 4m0s for CSIDriver "csi-mock-csi-mock-volumes-4249" May 15 01:46:33.784: INFO: waiting for CSIDriver csi-mock-csi-mock-volumes-4249 to register on node node1 STEP: Creating pod May 15 01:46:50.053: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil May 15 01:46:50.060: INFO: Waiting up to 5m0s for PersistentVolumeClaims [pvc-jg4zr] to have phase Bound May 15 01:46:50.062: INFO: PersistentVolumeClaim pvc-jg4zr found but phase is Pending instead of Bound. May 15 01:46:52.065: INFO: PersistentVolumeClaim pvc-jg4zr found and phase=Bound (2.004675586s) STEP: Expanding current pvc STEP: Waiting for persistent volume resize to finish STEP: Checking for conditions on pvc STEP: Deleting the previously created pod May 15 01:47:02.103: INFO: Deleting pod "pvc-volume-tester-dt6m6" in namespace "csi-mock-volumes-4249" May 15 01:47:02.109: INFO: Wait up to 5m0s for pod "pvc-volume-tester-dt6m6" to be fully deleted STEP: Creating a new pod with same volume STEP: Waiting for PVC resize to finish STEP: Deleting pod pvc-volume-tester-dt6m6 May 15 01:47:10.128: INFO: Deleting pod "pvc-volume-tester-dt6m6" in namespace "csi-mock-volumes-4249" STEP: Deleting pod pvc-volume-tester-9jznn May 15 01:47:10.130: INFO: Deleting pod "pvc-volume-tester-9jznn" in namespace "csi-mock-volumes-4249" May 15 01:47:10.134: INFO: Wait up to 5m0s for pod "pvc-volume-tester-9jznn" to be fully deleted STEP: Deleting claim pvc-jg4zr May 15 01:47:12.148: INFO: Waiting up to 2m0s for PersistentVolume pvc-274f6a25-abd6-4187-802c-656e870e4296 to get deleted May 15 01:47:12.150: INFO: PersistentVolume pvc-274f6a25-abd6-4187-802c-656e870e4296 found and phase=Bound (2.629103ms) May 15 01:47:14.154: INFO: PersistentVolume pvc-274f6a25-abd6-4187-802c-656e870e4296 was removed STEP: Deleting storageclass csi-mock-volumes-4249-sc STEP: Cleaning up resources STEP: deleting the test namespace: csi-mock-volumes-4249 STEP: Waiting for namespaces [csi-mock-volumes-4249] to vanish STEP: uninstalling csi mock driver May 15 01:47:20.173: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-4249-4927/csi-attacher May 15 01:47:20.178: INFO: deleting *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-4249 May 15 01:47:20.182: INFO: deleting *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-4249 May 15 01:47:20.186: INFO: deleting *v1.Role: csi-mock-volumes-4249-4927/external-attacher-cfg-csi-mock-volumes-4249 May 15 01:47:20.190: INFO: deleting *v1.RoleBinding: csi-mock-volumes-4249-4927/csi-attacher-role-cfg May 15 01:47:20.195: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-4249-4927/csi-provisioner May 15 01:47:20.200: INFO: deleting *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-4249 May 15 01:47:20.204: INFO: deleting *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-4249 May 15 01:47:20.208: INFO: deleting *v1.Role: csi-mock-volumes-4249-4927/external-provisioner-cfg-csi-mock-volumes-4249 May 15 01:47:20.213: INFO: deleting *v1.RoleBinding: csi-mock-volumes-4249-4927/csi-provisioner-role-cfg May 15 01:47:20.217: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-4249-4927/csi-resizer May 15 01:47:20.221: INFO: deleting *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-4249 May 15 01:47:20.225: INFO: deleting *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-4249 May 15 01:47:20.229: INFO: deleting *v1.Role: csi-mock-volumes-4249-4927/external-resizer-cfg-csi-mock-volumes-4249 May 15 01:47:20.233: INFO: deleting *v1.RoleBinding: csi-mock-volumes-4249-4927/csi-resizer-role-cfg May 15 01:47:20.237: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-4249-4927/csi-mock May 15 01:47:20.241: INFO: deleting *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-4249 May 15 01:47:20.245: INFO: deleting *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-4249 May 15 01:47:20.250: INFO: deleting *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-4249 May 15 01:47:20.254: INFO: deleting *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-4249 May 15 01:47:20.258: INFO: deleting *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-4249 May 15 01:47:20.263: INFO: deleting *v1.StorageClass: csi-mock-sc-csi-mock-volumes-4249 May 15 01:47:20.270: INFO: deleting *v1.StatefulSet: csi-mock-volumes-4249-4927/csi-mockplugin May 15 01:47:20.281: INFO: deleting *v1.CSIDriver: csi-mock-csi-mock-volumes-4249 May 15 01:47:20.293: INFO: deleting *v1.StatefulSet: csi-mock-volumes-4249-4927/csi-mockplugin-resizer STEP: deleting the driver namespace: csi-mock-volumes-4249-4927 STEP: Waiting for namespaces [csi-mock-volumes-4249-4927] to vanish [AfterEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:47:26.311: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "csi-mock-volumes-4249" for this suite. May 15 01:47:26.317: INFO: Namespace csi-mock-volumes-4249 was already deleted STEP: Destroying namespace "csi-mock-volumes-4249-4927" for this suite. May 15 01:47:26.320: INFO: Namespace csi-mock-volumes-4249-4927 was already deleted • [SLOW TEST:52.683 seconds] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 CSI Volume expansion /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:426 should expand volume by restarting pod if attach=off, nodeExpansion=on /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:455 ------------------------------ {"msg":"PASSED [sig-storage] CSI mock volume CSI Volume expansion should expand volume by restarting pod if attach=off, nodeExpansion=on","total":-1,"completed":4,"skipped":637,"failed":1,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] One pod requesting one prebound PVC should be able to mount volume and read from pod1"]} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:44:35.878: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename csi-mock-volumes STEP: Waiting for a default service account to be provisioned in namespace [It] should not expand volume if resizingOnDriver=off, resizingOnSC=on /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:455 STEP: Building a driver namespace object, basename csi-mock-volumes-3052 STEP: Waiting for a default service account to be provisioned in namespace STEP: deploying csi mock driver May 15 01:44:35.948: INFO: creating *v1.ServiceAccount: csi-mock-volumes-3052-9119/csi-attacher May 15 01:44:35.951: INFO: creating *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-3052 May 15 01:44:35.951: INFO: Define cluster role external-attacher-runner-csi-mock-volumes-3052 May 15 01:44:35.954: INFO: creating *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-3052 May 15 01:44:35.956: INFO: creating *v1.Role: csi-mock-volumes-3052-9119/external-attacher-cfg-csi-mock-volumes-3052 May 15 01:44:35.959: INFO: creating *v1.RoleBinding: csi-mock-volumes-3052-9119/csi-attacher-role-cfg May 15 01:44:35.961: INFO: creating *v1.ServiceAccount: csi-mock-volumes-3052-9119/csi-provisioner May 15 01:44:35.964: INFO: creating *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-3052 May 15 01:44:35.964: INFO: Define cluster role external-provisioner-runner-csi-mock-volumes-3052 May 15 01:44:35.967: INFO: creating *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-3052 May 15 01:44:35.969: INFO: creating *v1.Role: csi-mock-volumes-3052-9119/external-provisioner-cfg-csi-mock-volumes-3052 May 15 01:44:35.972: INFO: creating *v1.RoleBinding: csi-mock-volumes-3052-9119/csi-provisioner-role-cfg May 15 01:44:35.975: INFO: creating *v1.ServiceAccount: csi-mock-volumes-3052-9119/csi-resizer May 15 01:44:35.977: INFO: creating *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-3052 May 15 01:44:35.977: INFO: Define cluster role external-resizer-runner-csi-mock-volumes-3052 May 15 01:44:35.979: INFO: creating *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-3052 May 15 01:44:35.982: INFO: creating *v1.Role: csi-mock-volumes-3052-9119/external-resizer-cfg-csi-mock-volumes-3052 May 15 01:44:35.984: INFO: creating *v1.RoleBinding: csi-mock-volumes-3052-9119/csi-resizer-role-cfg May 15 01:44:35.988: INFO: creating *v1.ServiceAccount: csi-mock-volumes-3052-9119/csi-mock May 15 01:44:35.990: INFO: creating *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-3052 May 15 01:44:35.993: INFO: creating *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-3052 May 15 01:44:35.995: INFO: creating *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-3052 May 15 01:44:35.998: INFO: creating *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-3052 May 15 01:44:36.001: INFO: creating *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-3052 May 15 01:44:36.003: INFO: creating *v1.StorageClass: csi-mock-sc-csi-mock-volumes-3052 May 15 01:44:36.006: INFO: creating *v1.StatefulSet: csi-mock-volumes-3052-9119/csi-mockplugin May 15 01:44:36.010: INFO: creating *v1.StatefulSet: csi-mock-volumes-3052-9119/csi-mockplugin-attacher May 15 01:44:36.013: INFO: waiting for CSIDriver csi-mock-csi-mock-volumes-3052 to register on node node2 STEP: Creating pod May 15 01:44:45.532: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil May 15 01:44:45.539: INFO: Waiting up to 5m0s for PersistentVolumeClaims [pvc-nr42l] to have phase Bound May 15 01:44:45.542: INFO: PersistentVolumeClaim pvc-nr42l found but phase is Pending instead of Bound. May 15 01:44:47.545: INFO: PersistentVolumeClaim pvc-nr42l found and phase=Bound (2.005153579s) STEP: Expanding current pvc STEP: Deleting pod pvc-volume-tester-8nftk May 15 01:46:53.581: INFO: Deleting pod "pvc-volume-tester-8nftk" in namespace "csi-mock-volumes-3052" May 15 01:46:53.586: INFO: Wait up to 5m0s for pod "pvc-volume-tester-8nftk" to be fully deleted STEP: Deleting claim pvc-nr42l May 15 01:46:57.597: INFO: Waiting up to 2m0s for PersistentVolume pvc-a0bac380-54b8-4124-97d1-2403006def7b to get deleted May 15 01:46:57.599: INFO: PersistentVolume pvc-a0bac380-54b8-4124-97d1-2403006def7b found and phase=Bound (1.973912ms) May 15 01:46:59.601: INFO: PersistentVolume pvc-a0bac380-54b8-4124-97d1-2403006def7b found and phase=Released (2.004227879s) May 15 01:47:01.604: INFO: PersistentVolume pvc-a0bac380-54b8-4124-97d1-2403006def7b was removed STEP: Deleting storageclass csi-mock-volumes-3052-sc STEP: Cleaning up resources STEP: deleting the test namespace: csi-mock-volumes-3052 STEP: Waiting for namespaces [csi-mock-volumes-3052] to vanish STEP: uninstalling csi mock driver May 15 01:47:07.621: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-3052-9119/csi-attacher May 15 01:47:07.627: INFO: deleting *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-3052 May 15 01:47:07.631: INFO: deleting *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-3052 May 15 01:47:07.635: INFO: deleting *v1.Role: csi-mock-volumes-3052-9119/external-attacher-cfg-csi-mock-volumes-3052 May 15 01:47:07.639: INFO: deleting *v1.RoleBinding: csi-mock-volumes-3052-9119/csi-attacher-role-cfg May 15 01:47:07.644: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-3052-9119/csi-provisioner May 15 01:47:07.655: INFO: deleting *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-3052 May 15 01:47:07.666: INFO: deleting *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-3052 May 15 01:47:07.675: INFO: deleting *v1.Role: csi-mock-volumes-3052-9119/external-provisioner-cfg-csi-mock-volumes-3052 May 15 01:47:07.681: INFO: deleting *v1.RoleBinding: csi-mock-volumes-3052-9119/csi-provisioner-role-cfg May 15 01:47:07.696: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-3052-9119/csi-resizer May 15 01:47:07.701: INFO: deleting *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-3052 May 15 01:47:07.705: INFO: deleting *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-3052 May 15 01:47:07.709: INFO: deleting *v1.Role: csi-mock-volumes-3052-9119/external-resizer-cfg-csi-mock-volumes-3052 May 15 01:47:07.713: INFO: deleting *v1.RoleBinding: csi-mock-volumes-3052-9119/csi-resizer-role-cfg May 15 01:47:07.717: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-3052-9119/csi-mock May 15 01:47:07.721: INFO: deleting *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-3052 May 15 01:47:07.725: INFO: deleting *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-3052 May 15 01:47:07.729: INFO: deleting *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-3052 May 15 01:47:07.734: INFO: deleting *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-3052 May 15 01:47:07.739: INFO: deleting *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-3052 May 15 01:47:07.743: INFO: deleting *v1.StorageClass: csi-mock-sc-csi-mock-volumes-3052 May 15 01:47:07.748: INFO: deleting *v1.StatefulSet: csi-mock-volumes-3052-9119/csi-mockplugin May 15 01:47:07.753: INFO: deleting *v1.StatefulSet: csi-mock-volumes-3052-9119/csi-mockplugin-attacher STEP: deleting the driver namespace: csi-mock-volumes-3052-9119 STEP: Waiting for namespaces [csi-mock-volumes-3052-9119] to vanish [AfterEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:47:35.766: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "csi-mock-volumes-3052" for this suite. May 15 01:47:35.773: INFO: Namespace csi-mock-volumes-3052 was already deleted STEP: Destroying namespace "csi-mock-volumes-3052-9119" for this suite. May 15 01:47:35.776: INFO: Namespace csi-mock-volumes-3052-9119 was already deleted • [SLOW TEST:179.898 seconds] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 CSI Volume expansion /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:426 should not expand volume if resizingOnDriver=off, resizingOnSC=on /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:455 ------------------------------ {"msg":"PASSED [sig-storage] CSI mock volume CSI Volume expansion should not expand volume if resizingOnDriver=off, resizingOnSC=on","total":-1,"completed":2,"skipped":380,"failed":0} SSSSSSS ------------------------------ [BeforeEach] [sig-storage] PV Protection /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:47:35.795: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename pv-protection STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PV Protection /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/pv_protection.go:51 May 15 01:47:35.814: INFO: Waiting up to 30m0s for all (but 0) nodes to be schedulable STEP: Creating a PV STEP: Waiting for PV to enter phase Available May 15 01:47:35.819: INFO: Waiting up to 30s for PersistentVolume hostpath-gjrm2 to have phase Available May 15 01:47:35.821: INFO: PersistentVolume hostpath-gjrm2 found but phase is Pending instead of Available. May 15 01:47:36.824: INFO: PersistentVolume hostpath-gjrm2 found and phase=Available (1.005487793s) STEP: Checking that PV Protection finalizer is set [It] Verify that PV bound to a PVC is not removed immediately /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/pv_protection.go:107 STEP: Creating a PVC STEP: Waiting for PVC to become Bound May 15 01:47:36.832: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-bmfzj] to have phase Bound May 15 01:47:36.834: INFO: PersistentVolumeClaim pvc-bmfzj found but phase is Pending instead of Bound. May 15 01:47:38.838: INFO: PersistentVolumeClaim pvc-bmfzj found and phase=Bound (2.005864686s) STEP: Deleting the PV, however, the PV must not be removed from the system as it's bound to a PVC STEP: Checking that the PV status is Terminating STEP: Deleting the PVC that is bound to the PV STEP: Checking that the PV is automatically removed from the system because it's no longer bound to a PVC May 15 01:47:38.848: INFO: Waiting up to 3m0s for PersistentVolume hostpath-gjrm2 to get deleted May 15 01:47:38.850: INFO: PersistentVolume hostpath-gjrm2 found and phase=Bound (2.005371ms) May 15 01:47:40.853: INFO: PersistentVolume hostpath-gjrm2 was removed [AfterEach] [sig-storage] PV Protection /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:47:40.853: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "pv-protection-550" for this suite. [AfterEach] [sig-storage] PV Protection /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/pv_protection.go:92 May 15 01:47:40.863: INFO: AfterEach: Cleaning up test resources. May 15 01:47:40.863: INFO: Deleting PersistentVolumeClaim "pvc-bmfzj" May 15 01:47:40.865: INFO: Deleting PersistentVolume "hostpath-gjrm2" • [SLOW TEST:5.072 seconds] [sig-storage] PV Protection /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 Verify that PV bound to a PVC is not removed immediately /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/pv_protection.go:107 ------------------------------ {"msg":"PASSED [sig-storage] PV Protection Verify that PV bound to a PVC is not removed immediately","total":-1,"completed":3,"skipped":387,"failed":0} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:46:31.037: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename csi-mock-volumes STEP: Waiting for a default service account to be provisioned in namespace [It] should require VolumeAttach for drivers with attachment /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:274 STEP: Building a driver namespace object, basename csi-mock-volumes-9809 STEP: Waiting for a default service account to be provisioned in namespace STEP: deploying csi mock driver May 15 01:46:31.109: INFO: creating *v1.ServiceAccount: csi-mock-volumes-9809-929/csi-attacher May 15 01:46:31.112: INFO: creating *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-9809 May 15 01:46:31.112: INFO: Define cluster role external-attacher-runner-csi-mock-volumes-9809 May 15 01:46:31.115: INFO: creating *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-9809 May 15 01:46:31.118: INFO: creating *v1.Role: csi-mock-volumes-9809-929/external-attacher-cfg-csi-mock-volumes-9809 May 15 01:46:31.120: INFO: creating *v1.RoleBinding: csi-mock-volumes-9809-929/csi-attacher-role-cfg May 15 01:46:31.123: INFO: creating *v1.ServiceAccount: csi-mock-volumes-9809-929/csi-provisioner May 15 01:46:31.126: INFO: creating *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-9809 May 15 01:46:31.126: INFO: Define cluster role external-provisioner-runner-csi-mock-volumes-9809 May 15 01:46:31.129: INFO: creating *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-9809 May 15 01:46:31.132: INFO: creating *v1.Role: csi-mock-volumes-9809-929/external-provisioner-cfg-csi-mock-volumes-9809 May 15 01:46:31.135: INFO: creating *v1.RoleBinding: csi-mock-volumes-9809-929/csi-provisioner-role-cfg May 15 01:46:31.138: INFO: creating *v1.ServiceAccount: csi-mock-volumes-9809-929/csi-resizer May 15 01:46:31.140: INFO: creating *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-9809 May 15 01:46:31.140: INFO: Define cluster role external-resizer-runner-csi-mock-volumes-9809 May 15 01:46:31.142: INFO: creating *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-9809 May 15 01:46:31.145: INFO: creating *v1.Role: csi-mock-volumes-9809-929/external-resizer-cfg-csi-mock-volumes-9809 May 15 01:46:31.148: INFO: creating *v1.RoleBinding: csi-mock-volumes-9809-929/csi-resizer-role-cfg May 15 01:46:31.150: INFO: creating *v1.ServiceAccount: csi-mock-volumes-9809-929/csi-mock May 15 01:46:31.152: INFO: creating *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-9809 May 15 01:46:31.155: INFO: creating *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-9809 May 15 01:46:31.157: INFO: creating *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-9809 May 15 01:46:31.160: INFO: creating *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-9809 May 15 01:46:31.163: INFO: creating *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-9809 May 15 01:46:31.165: INFO: creating *v1.StorageClass: csi-mock-sc-csi-mock-volumes-9809 May 15 01:46:31.168: INFO: creating *v1.StatefulSet: csi-mock-volumes-9809-929/csi-mockplugin May 15 01:46:31.172: INFO: creating *v1.CSIDriver: csi-mock-csi-mock-volumes-9809 May 15 01:46:31.176: INFO: creating *v1.StatefulSet: csi-mock-volumes-9809-929/csi-mockplugin-attacher May 15 01:46:31.180: INFO: waiting up to 4m0s for CSIDriver "csi-mock-csi-mock-volumes-9809" May 15 01:46:31.181: INFO: waiting for CSIDriver csi-mock-csi-mock-volumes-9809 to register on node node1 STEP: Creating pod May 15 01:46:47.451: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil May 15 01:46:47.458: INFO: Waiting up to 5m0s for PersistentVolumeClaims [pvc-srhpx] to have phase Bound May 15 01:46:47.461: INFO: PersistentVolumeClaim pvc-srhpx found but phase is Pending instead of Bound. May 15 01:46:49.464: INFO: PersistentVolumeClaim pvc-srhpx found and phase=Bound (2.006161204s) STEP: Checking if VolumeAttachment was created for the pod STEP: Deleting pod pvc-volume-tester-hp7xk May 15 01:46:59.496: INFO: Deleting pod "pvc-volume-tester-hp7xk" in namespace "csi-mock-volumes-9809" May 15 01:46:59.501: INFO: Wait up to 5m0s for pod "pvc-volume-tester-hp7xk" to be fully deleted STEP: Deleting claim pvc-srhpx May 15 01:47:07.514: INFO: Waiting up to 2m0s for PersistentVolume pvc-1a14b718-4147-4f0f-b127-415fc6d436fc to get deleted May 15 01:47:07.517: INFO: PersistentVolume pvc-1a14b718-4147-4f0f-b127-415fc6d436fc found and phase=Bound (2.456602ms) May 15 01:47:09.520: INFO: PersistentVolume pvc-1a14b718-4147-4f0f-b127-415fc6d436fc was removed STEP: Deleting storageclass csi-mock-volumes-9809-sc STEP: Cleaning up resources STEP: deleting the test namespace: csi-mock-volumes-9809 STEP: Waiting for namespaces [csi-mock-volumes-9809] to vanish STEP: uninstalling csi mock driver May 15 01:47:15.542: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-9809-929/csi-attacher May 15 01:47:15.549: INFO: deleting *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-9809 May 15 01:47:15.554: INFO: deleting *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-9809 May 15 01:47:15.561: INFO: deleting *v1.Role: csi-mock-volumes-9809-929/external-attacher-cfg-csi-mock-volumes-9809 May 15 01:47:15.565: INFO: deleting *v1.RoleBinding: csi-mock-volumes-9809-929/csi-attacher-role-cfg May 15 01:47:15.569: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-9809-929/csi-provisioner May 15 01:47:15.574: INFO: deleting *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-9809 May 15 01:47:15.578: INFO: deleting *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-9809 May 15 01:47:15.582: INFO: deleting *v1.Role: csi-mock-volumes-9809-929/external-provisioner-cfg-csi-mock-volumes-9809 May 15 01:47:15.587: INFO: deleting *v1.RoleBinding: csi-mock-volumes-9809-929/csi-provisioner-role-cfg May 15 01:47:15.591: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-9809-929/csi-resizer May 15 01:47:15.597: INFO: deleting *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-9809 May 15 01:47:15.600: INFO: deleting *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-9809 May 15 01:47:15.604: INFO: deleting *v1.Role: csi-mock-volumes-9809-929/external-resizer-cfg-csi-mock-volumes-9809 May 15 01:47:15.609: INFO: deleting *v1.RoleBinding: csi-mock-volumes-9809-929/csi-resizer-role-cfg May 15 01:47:15.613: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-9809-929/csi-mock May 15 01:47:15.618: INFO: deleting *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-9809 May 15 01:47:15.622: INFO: deleting *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-9809 May 15 01:47:15.626: INFO: deleting *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-9809 May 15 01:47:15.630: INFO: deleting *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-9809 May 15 01:47:15.634: INFO: deleting *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-9809 May 15 01:47:15.639: INFO: deleting *v1.StorageClass: csi-mock-sc-csi-mock-volumes-9809 May 15 01:47:15.645: INFO: deleting *v1.StatefulSet: csi-mock-volumes-9809-929/csi-mockplugin May 15 01:47:15.650: INFO: deleting *v1.CSIDriver: csi-mock-csi-mock-volumes-9809 May 15 01:47:15.654: INFO: deleting *v1.StatefulSet: csi-mock-volumes-9809-929/csi-mockplugin-attacher STEP: deleting the driver namespace: csi-mock-volumes-9809-929 STEP: Waiting for namespaces [csi-mock-volumes-9809-929] to vanish [AfterEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:47:43.668: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "csi-mock-volumes-9809" for this suite. May 15 01:47:43.675: INFO: Namespace csi-mock-volumes-9809 was already deleted STEP: Destroying namespace "csi-mock-volumes-9809-929" for this suite. May 15 01:47:43.678: INFO: Namespace csi-mock-volumes-9809-929 was already deleted • [SLOW TEST:72.641 seconds] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 CSI attach test using mock driver /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:252 should require VolumeAttach for drivers with attachment /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:274 ------------------------------ {"msg":"PASSED [sig-storage] CSI mock volume CSI attach test using mock driver should require VolumeAttach for drivers with attachment","total":-1,"completed":2,"skipped":77,"failed":1,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2"]} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] HostPath /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:47:41.048: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename hostpath STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] HostPath /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/host_path.go:37 [It] should give a volume the correct mode [LinuxOnly] [NodeConformance] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/host_path.go:48 STEP: Creating a pod to test hostPath mode May 15 01:47:41.079: INFO: Waiting up to 5m0s for pod "pod-host-path-test" in namespace "hostpath-4854" to be "Succeeded or Failed" May 15 01:47:41.081: INFO: Pod "pod-host-path-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2.113874ms May 15 01:47:43.084: INFO: Pod "pod-host-path-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2.005062158s May 15 01:47:45.088: INFO: Pod "pod-host-path-test": Phase="Pending", Reason="", readiness=false. Elapsed: 4.008349031s May 15 01:47:47.091: INFO: Pod "pod-host-path-test": Phase="Succeeded", Reason="", readiness=false. Elapsed: 6.012170756s STEP: Saw pod success May 15 01:47:47.091: INFO: Pod "pod-host-path-test" satisfied condition "Succeeded or Failed" May 15 01:47:47.094: INFO: Trying to get logs from node node2 pod pod-host-path-test container test-container-1: STEP: delete the pod May 15 01:47:47.109: INFO: Waiting for pod pod-host-path-test to disappear May 15 01:47:47.111: INFO: Pod pod-host-path-test no longer exists [AfterEach] [sig-storage] HostPath /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:47:47.111: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "hostpath-4854" for this suite. • [SLOW TEST:6.072 seconds] [sig-storage] HostPath /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/host_path.go:34 should give a volume the correct mode [LinuxOnly] [NodeConformance] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/host_path.go:48 ------------------------------ {"msg":"PASSED [sig-storage] HostPath should give a volume the correct mode [LinuxOnly] [NodeConformance]","total":-1,"completed":4,"skipped":483,"failed":0} SSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] GCP Volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:47:47.174: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename gcp-volume STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] GCP Volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/volumes.go:69 May 15 01:47:47.194: INFO: Only supported for node OS distro [gci ubuntu custom] (not debian) [AfterEach] [sig-storage] GCP Volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:47:47.195: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "gcp-volume-5345" for this suite. S [SKIPPING] in Spec Setup (BeforeEach) [0.030 seconds] [sig-storage] GCP Volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/volumes.go:61 NFSv4 [BeforeEach] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/volumes.go:79 should be mountable for NFSv4 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/volumes.go:80 Only supported for node OS distro [gci ubuntu custom] (not debian) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/volumes.go:70 ------------------------------ S ------------------------------ [BeforeEach] [sig-storage] Ephemeralstorage /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:47:26.391: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename pv STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] Ephemeralstorage /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/ephemeral_volume.go:49 [It] should allow deletion of pod with invalid volume : secret /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/ephemeral_volume.go:55 May 15 01:47:56.425: INFO: Deleting pod "pv-210"/"pod-ephm-test-projected-26wt" May 15 01:47:56.425: INFO: Deleting pod "pod-ephm-test-projected-26wt" in namespace "pv-210" May 15 01:47:56.430: INFO: Wait up to 5m0s for pod "pod-ephm-test-projected-26wt" to be fully deleted [AfterEach] [sig-storage] Ephemeralstorage /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:48:00.436: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "pv-210" for this suite. • [SLOW TEST:34.053 seconds] [sig-storage] Ephemeralstorage /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 When pod refers to non-existent ephemeral storage /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/ephemeral_volume.go:53 should allow deletion of pod with invalid volume : secret /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/ephemeral_volume.go:55 ------------------------------ {"msg":"PASSED [sig-storage] Ephemeralstorage When pod refers to non-existent ephemeral storage should allow deletion of pod with invalid volume : secret","total":-1,"completed":5,"skipped":670,"failed":1,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] One pod requesting one prebound PVC should be able to mount volume and read from pod1"]} SSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:46:14.523: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: dir-link-bindmounted] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes May 15 01:46:18.578: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir /tmp/local-volume-test-62fcea39-32f1-48d4-afbf-213c4d7e9040-backend && mount --bind /tmp/local-volume-test-62fcea39-32f1-48d4-afbf-213c4d7e9040-backend /tmp/local-volume-test-62fcea39-32f1-48d4-afbf-213c4d7e9040-backend && ln -s /tmp/local-volume-test-62fcea39-32f1-48d4-afbf-213c4d7e9040-backend /tmp/local-volume-test-62fcea39-32f1-48d4-afbf-213c4d7e9040] Namespace:persistent-local-volumes-test-6862 PodName:hostexec-node1-lmdw4 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:46:18.578: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:46:18.701: INFO: Creating a PV followed by a PVC May 15 01:46:18.708: INFO: Waiting for PV local-pvphpfw to bind to PVC pvc-vzpdt May 15 01:46:18.708: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-vzpdt] to have phase Bound May 15 01:46:18.711: INFO: PersistentVolumeClaim pvc-vzpdt found but phase is Pending instead of Bound. May 15 01:46:20.715: INFO: PersistentVolumeClaim pvc-vzpdt found and phase=Bound (2.007033031s) May 15 01:46:20.715: INFO: Waiting up to 3m0s for PersistentVolume local-pvphpfw to have phase Bound May 15 01:46:20.718: INFO: PersistentVolume local-pvphpfw found and phase=Bound (2.88551ms) [BeforeEach] One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:211 STEP: Creating pod1 STEP: Creating a pod May 15 01:48:20.745: FAIL: Unexpected error: <*errors.errorString | 0xc000c44120>: { s: "pod \"pod-c61f2051-9957-419e-995d-c3ed08d36192\" is not Running: timed out waiting for the condition", } pod "pod-c61f2051-9957-419e-995d-c3ed08d36192" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage.glob..func20.2.3.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 +0x119 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000703800) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc000703800) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc000703800, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:223 STEP: Deleting pod1 STEP: Deleting pod pod-c61f2051-9957-419e-995d-c3ed08d36192 in namespace persistent-local-volumes-test-6862 [AfterEach] [Volume type: dir-link-bindmounted] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:48:20.753: INFO: Deleting PersistentVolumeClaim "pvc-vzpdt" May 15 01:48:20.757: INFO: Deleting PersistentVolume "local-pvphpfw" STEP: Removing the test directory May 15 01:48:20.761: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm /tmp/local-volume-test-62fcea39-32f1-48d4-afbf-213c4d7e9040 && umount /tmp/local-volume-test-62fcea39-32f1-48d4-afbf-213c4d7e9040-backend && rm -r /tmp/local-volume-test-62fcea39-32f1-48d4-afbf-213c4d7e9040-backend] Namespace:persistent-local-volumes-test-6862 PodName:hostexec-node1-lmdw4 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:48:20.761: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "persistent-local-volumes-test-6862". STEP: Found 12 events. May 15 01:48:20.905: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for hostexec-node1-lmdw4: { } Scheduled: Successfully assigned persistent-local-volumes-test-6862/hostexec-node1-lmdw4 to node1 May 15 01:48:20.905: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-c61f2051-9957-419e-995d-c3ed08d36192: { } Scheduled: Successfully assigned persistent-local-volumes-test-6862/pod-c61f2051-9957-419e-995d-c3ed08d36192 to node1 May 15 01:48:20.905: INFO: At 2021-05-15 01:46:16 +0000 UTC - event for hostexec-node1-lmdw4: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 15 01:48:20.905: INFO: At 2021-05-15 01:46:17 +0000 UTC - event for hostexec-node1-lmdw4: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 1.254379498s May 15 01:48:20.905: INFO: At 2021-05-15 01:46:17 +0000 UTC - event for hostexec-node1-lmdw4: {kubelet node1} Created: Created container agnhost-container May 15 01:48:20.905: INFO: At 2021-05-15 01:46:17 +0000 UTC - event for hostexec-node1-lmdw4: {kubelet node1} Started: Started container agnhost-container May 15 01:48:20.905: INFO: At 2021-05-15 01:46:23 +0000 UTC - event for pod-c61f2051-9957-419e-995d-c3ed08d36192: {multus } AddedInterface: Add eth0 [10.244.3.140/24] May 15 01:48:20.905: INFO: At 2021-05-15 01:46:23 +0000 UTC - event for pod-c61f2051-9957-419e-995d-c3ed08d36192: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:48:20.905: INFO: At 2021-05-15 01:46:24 +0000 UTC - event for pod-c61f2051-9957-419e-995d-c3ed08d36192: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:48:20.905: INFO: At 2021-05-15 01:46:24 +0000 UTC - event for pod-c61f2051-9957-419e-995d-c3ed08d36192: {kubelet node1} Failed: Error: ErrImagePull May 15 01:48:20.905: INFO: At 2021-05-15 01:46:24 +0000 UTC - event for pod-c61f2051-9957-419e-995d-c3ed08d36192: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 15 01:48:20.905: INFO: At 2021-05-15 01:46:24 +0000 UTC - event for pod-c61f2051-9957-419e-995d-c3ed08d36192: {kubelet node1} Failed: Error: ImagePullBackOff May 15 01:48:20.908: INFO: POD NODE PHASE GRACE CONDITIONS May 15 01:48:20.908: INFO: hostexec-node1-lmdw4 node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:14 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:18 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:18 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:14 +0000 UTC }] May 15 01:48:20.908: INFO: pod-c61f2051-9957-419e-995d-c3ed08d36192 node1 Pending 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:20 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:20 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:20 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:20 +0000 UTC }] May 15 01:48:20.908: INFO: May 15 01:48:20.911: INFO: Logging node info for node master1 May 15 01:48:20.914: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 156264 0 2021-05-14 19:56:35 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:18 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:18 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:18 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:48:18 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[@ :],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:48:20.915: INFO: Logging kubelet events for node master1 May 15 01:48:20.918: INFO: Logging pods the kubelet thinks is on node master1 May 15 01:48:20.931: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:48:20.931: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:48:20.931: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:48:20.931: INFO: Init container install-cni ready: true, restart count 0 May 15 01:48:20.931: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:48:20.931: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:48:20.931: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:48:20.931: INFO: Container node-exporter ready: true, restart count 0 May 15 01:48:20.931: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:48:20.931: INFO: Container kube-scheduler ready: true, restart count 0 May 15 01:48:20.931: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 15 01:48:20.931: INFO: Container docker-registry ready: true, restart count 0 May 15 01:48:20.931: INFO: Container nginx ready: true, restart count 0 May 15 01:48:20.931: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 15 01:48:20.931: INFO: Container nfd-controller ready: true, restart count 0 May 15 01:48:20.931: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:48:20.931: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:48:20.931: INFO: kubernetes-metrics-scraper-678c97765c-fswrn started at 2021-05-15 00:18:49 +0000 UTC (0+1 container statuses recorded) May 15 01:48:20.931: INFO: Container kubernetes-metrics-scraper ready: true, restart count 0 May 15 01:48:20.931: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:48:20.931: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:48:20.931: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:48:20.931: INFO: Container kube-multus ready: true, restart count 1 W0515 01:48:20.945334 34 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:48:20.974: INFO: Latency metrics for node master1 May 15 01:48:20.974: INFO: Logging node info for node master2 May 15 01:48:20.977: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 156247 0 2021-05-14 19:57:04 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:16 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:16 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:16 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:48:16 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:48:20.977: INFO: Logging kubelet events for node master2 May 15 01:48:20.983: INFO: Logging pods the kubelet thinks is on node master2 May 15 01:48:20.996: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:48:20.996: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:48:20.996: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:48:20.996: INFO: Container kube-multus ready: true, restart count 1 May 15 01:48:20.996: INFO: coredns-7677f9bb54-96w24 started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:48:20.996: INFO: Container coredns ready: true, restart count 0 May 15 01:48:20.996: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 15 01:48:20.996: INFO: Container autoscaler ready: true, restart count 2 May 15 01:48:20.996: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:48:20.996: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:48:20.996: INFO: Container node-exporter ready: true, restart count 0 May 15 01:48:20.996: INFO: prometheus-operator-5bb8cb9d8f-fqb87 started at 2021-05-15 00:18:49 +0000 UTC (0+2 container statuses recorded) May 15 01:48:20.996: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:48:20.996: INFO: Container prometheus-operator ready: true, restart count 0 May 15 01:48:20.996: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:48:20.996: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:48:20.996: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:48:20.996: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:48:20.996: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:48:20.996: INFO: Container kube-scheduler ready: true, restart count 2 May 15 01:48:20.996: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:48:20.996: INFO: Init container install-cni ready: true, restart count 0 May 15 01:48:20.996: INFO: Container kube-flannel ready: true, restart count 1 W0515 01:48:21.007715 34 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:48:21.034: INFO: Latency metrics for node master2 May 15 01:48:21.034: INFO: Logging node info for node master3 May 15 01:48:21.036: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 156226 0 2021-05-14 19:57:14 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234767872 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324603904 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:14 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:14 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:14 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:48:14 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:48:21.036: INFO: Logging kubelet events for node master3 May 15 01:48:21.041: INFO: Logging pods the kubelet thinks is on node master3 May 15 01:48:21.054: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.054: INFO: Container kube-scheduler ready: true, restart count 3 May 15 01:48:21.054: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.054: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:48:21.054: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.054: INFO: Container kube-proxy ready: true, restart count 1 May 15 01:48:21.054: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:48:21.054: INFO: Init container install-cni ready: true, restart count 0 May 15 01:48:21.054: INFO: Container kube-flannel ready: true, restart count 2 May 15 01:48:21.054: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.054: INFO: Container kube-multus ready: true, restart count 1 May 15 01:48:21.054: INFO: coredns-7677f9bb54-rpj8c started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.054: INFO: Container coredns ready: true, restart count 0 May 15 01:48:21.054: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:48:21.054: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:48:21.054: INFO: Container node-exporter ready: true, restart count 0 May 15 01:48:21.054: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.054: INFO: Container kube-controller-manager ready: true, restart count 3 W0515 01:48:21.067732 34 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:48:21.094: INFO: Latency metrics for node master3 May 15 01:48:21.094: INFO: Logging node info for node node1 May 15 01:48:21.098: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 156227 0 2021-05-14 19:58:21 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-7734":"csi-mock-csi-mock-volumes-7734","csi-mock-csi-mock-volumes-9728":"csi-mock-csi-mock-volumes-9728","csi-mock-csi-mock-volumes-9809":"csi-mock-csi-mock-volumes-9809"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:00 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:46:49 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:47:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:14 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:14 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:14 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:48:14 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[@ :],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:48:21.099: INFO: Logging kubelet events for node node1 May 15 01:48:21.103: INFO: Logging pods the kubelet thinks is on node node1 May 15 01:48:21.125: INFO: hostexec-node1-sfsdm started at 2021-05-15 01:47:47 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.125: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:21.125: INFO: hostexec-node1-sv6vb started at 2021-05-15 01:46:26 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.125: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:21.125: INFO: pod-bec58502-0df0-4903-a0f2-b7940a229fd0 started at 2021-05-15 01:47:58 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.125: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:21.125: INFO: hostexec-node1-b5m77 started at 2021-05-15 01:48:00 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.125: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:21.125: INFO: pod-604ae13e-a2a4-44cd-87c9-c2aa1f39edc0 started at 2021-05-15 01:46:28 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.125: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:21.125: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:48:21.125: INFO: Init container install-cni ready: true, restart count 0 May 15 01:48:21.125: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:48:21.125: INFO: node-feature-discovery-worker-bw8zg started at 2021-05-15 00:18:56 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.125: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:48:21.125: INFO: collectd-mrzps started at 2021-05-15 00:19:22 +0000 UTC (0+3 container statuses recorded) May 15 01:48:21.125: INFO: Container collectd ready: true, restart count 0 May 15 01:48:21.125: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:48:21.125: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:48:21.125: INFO: hostexec-node1-lmdw4 started at 2021-05-15 01:46:14 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.125: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:21.126: INFO: hostexec-node1-5nkl7 started at 2021-05-15 01:46:52 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.126: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:21.126: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.126: INFO: Container kube-multus ready: true, restart count 1 May 15 01:48:21.126: INFO: prometheus-k8s-0 started at 2021-05-15 00:19:01 +0000 UTC (0+5 container statuses recorded) May 15 01:48:21.126: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 15 01:48:21.126: INFO: Container grafana ready: true, restart count 0 May 15 01:48:21.126: INFO: Container prometheus ready: true, restart count 26 May 15 01:48:21.126: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 15 01:48:21.126: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 15 01:48:21.126: INFO: csi-mockplugin-0 started at 2021-05-15 01:47:22 +0000 UTC (0+3 container statuses recorded) May 15 01:48:21.126: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:48:21.126: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:48:21.126: INFO: Container mock ready: true, restart count 0 May 15 01:48:21.126: INFO: hostexec-node1-5fw6h started at 2021-05-15 01:46:22 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.126: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:21.126: INFO: pod-6c919e34-2cba-46c9-86ed-54b69dfbc370 started at 2021-05-15 01:47:59 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.126: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:21.126: INFO: pod-c3357cc4-fc25-4578-af0f-b315e3d2cb0c started at 2021-05-15 01:48:14 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.126: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:21.126: INFO: node-exporter-flvqz started at 2021-05-15 00:18:55 +0000 UTC (0+2 container statuses recorded) May 15 01:48:21.126: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:48:21.126: INFO: Container node-exporter ready: true, restart count 0 May 15 01:48:21.126: INFO: pvc-volume-tester-hwvvq started at 2021-05-15 01:47:34 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.126: INFO: Container volume-tester ready: true, restart count 0 May 15 01:48:21.126: INFO: pod-c61f2051-9957-419e-995d-c3ed08d36192 started at 2021-05-15 01:46:20 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.126: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:21.126: INFO: hostexec-node1-xz74m started at 2021-05-15 01:47:43 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.126: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:21.126: INFO: hostexec-node1-r82s6 started at 2021-05-15 01:46:27 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.126: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:21.126: INFO: cmk-4s6dm started at 2021-05-15 00:18:54 +0000 UTC (0+2 container statuses recorded) May 15 01:48:21.126: INFO: Container nodereport ready: true, restart count 0 May 15 01:48:21.126: INFO: Container reconcile ready: true, restart count 0 May 15 01:48:21.126: INFO: pod-470e75a2-4056-4cd4-838e-f67be7033ced started at 2021-05-15 01:46:43 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.126: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:21.126: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.126: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:48:21.126: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4vxcc started at 2021-05-15 00:19:00 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.126: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:48:21.126: INFO: hostexec-node1-frmv7 started at 2021-05-15 01:46:31 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.126: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:21.126: INFO: pod-dbfe8699-c076-46f1-bf35-86df11b4addd started at 2021-05-15 01:46:42 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.126: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:21.126: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.126: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:48:21.126: INFO: pod-4cdc198f-2eab-46aa-b552-61f5a2bf0de0 started at 2021-05-15 01:46:33 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.126: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:21.126: INFO: pod-a44304e0-d8e2-42b6-8188-544b7d09e42b started at 2021-05-15 01:47:14 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.126: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:21.126: INFO: csi-mockplugin-resizer-0 started at 2021-05-15 01:47:22 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.126: INFO: Container csi-resizer ready: true, restart count 0 W0515 01:48:21.138994 34 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:48:21.175: INFO: Latency metrics for node node1 May 15 01:48:21.175: INFO: Logging node info for node node2 May 15 01:48:21.177: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 156280 0 2021-05-14 19:58:22 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-3052":"csi-mock-csi-mock-volumes-3052","csi-mock-csi-mock-volumes-8760":"csi-mock-csi-mock-volumes-8760","csi-mock-csi-mock-volumes-9624":"csi-mock-csi-mock-volumes-9624"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:46 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:44:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:46:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:io.kubernetes.storage.mock/node":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:19 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:19 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:19 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:48:19 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:48:21.178: INFO: Logging kubelet events for node node2 May 15 01:48:21.183: INFO: Logging pods the kubelet thinks is on node node2 May 15 01:48:21.193: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.193: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:48:21.193: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 15 01:48:21.193: INFO: Container nodereport ready: true, restart count 0 May 15 01:48:21.193: INFO: Container reconcile ready: true, restart count 0 May 15 01:48:21.193: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.193: INFO: Container cmk-webhook ready: true, restart count 0 May 15 01:48:21.193: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 15 01:48:21.193: INFO: Container discover ready: false, restart count 0 May 15 01:48:21.193: INFO: Container init ready: false, restart count 0 May 15 01:48:21.193: INFO: Container install ready: false, restart count 0 May 15 01:48:21.193: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.193: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:48:21.193: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.193: INFO: Container kube-multus ready: true, restart count 1 May 15 01:48:21.193: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.193: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 15 01:48:21.193: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.193: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:48:21.193: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:48:21.193: INFO: Init container install-cni ready: true, restart count 1 May 15 01:48:21.193: INFO: Container kube-flannel ready: true, restart count 4 May 15 01:48:21.193: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:48:21.193: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:48:21.193: INFO: Container node-exporter ready: true, restart count 0 May 15 01:48:21.193: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 15 01:48:21.193: INFO: Container tas-controller ready: true, restart count 0 May 15 01:48:21.193: INFO: Container tas-extender ready: true, restart count 0 May 15 01:48:21.193: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 15 01:48:21.193: INFO: Container collectd ready: true, restart count 0 May 15 01:48:21.193: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:48:21.193: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:48:21.193: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 15 01:48:21.193: INFO: Container kube-sriovdp ready: true, restart count 0 W0515 01:48:21.205612 34 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:48:21.251: INFO: Latency metrics for node node2 May 15 01:48:21.251: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-6862" for this suite. • Failure in Spec Setup (BeforeEach) [126.737 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: dir-link-bindmounted] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:205 should be able to mount volume and read from pod1 [BeforeEach] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:228 May 15 01:48:20.745: Unexpected error: <*errors.errorString | 0xc000c44120>: { s: "pod \"pod-c61f2051-9957-419e-995d-c3ed08d36192\" is not Running: timed out waiting for the condition", } pod "pod-c61f2051-9957-419e-995d-c3ed08d36192" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 ------------------------------ {"msg":"FAILED [sig-storage] PersistentVolumes-local [Volume type: dir-link-bindmounted] One pod requesting one prebound PVC should be able to mount volume and read from pod1","total":-1,"completed":0,"skipped":53,"failed":2,"failures":["[sig-storage] PersistentVolumes-local [Volume type: blockfswithoutformat] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir-link-bindmounted] One pod requesting one prebound PVC should be able to mount volume and read from pod1"]} SSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:46:22.231: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: dir-link] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes May 15 01:46:26.274: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir /tmp/local-volume-test-f56d2ae0-da77-4125-a06b-cae0ee778e1c-backend && ln -s /tmp/local-volume-test-f56d2ae0-da77-4125-a06b-cae0ee778e1c-backend /tmp/local-volume-test-f56d2ae0-da77-4125-a06b-cae0ee778e1c] Namespace:persistent-local-volumes-test-276 PodName:hostexec-node1-5fw6h ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:46:26.274: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:46:26.397: INFO: Creating a PV followed by a PVC May 15 01:46:26.403: INFO: Waiting for PV local-pvn2wh2 to bind to PVC pvc-qptpw May 15 01:46:26.403: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-qptpw] to have phase Bound May 15 01:46:26.405: INFO: PersistentVolumeClaim pvc-qptpw found but phase is Pending instead of Bound. May 15 01:46:28.409: INFO: PersistentVolumeClaim pvc-qptpw found and phase=Bound (2.006150235s) May 15 01:46:28.409: INFO: Waiting up to 3m0s for PersistentVolume local-pvn2wh2 to have phase Bound May 15 01:46:28.411: INFO: PersistentVolume local-pvn2wh2 found and phase=Bound (2.186126ms) [BeforeEach] One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:211 STEP: Creating pod1 STEP: Creating a pod May 15 01:48:28.433: FAIL: Unexpected error: <*errors.errorString | 0xc0009708b0>: { s: "pod \"pod-604ae13e-a2a4-44cd-87c9-c2aa1f39edc0\" is not Running: timed out waiting for the condition", } pod "pod-604ae13e-a2a4-44cd-87c9-c2aa1f39edc0" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage.glob..func20.2.3.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 +0x119 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc003230480) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc003230480) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc003230480, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:223 STEP: Deleting pod1 STEP: Deleting pod pod-604ae13e-a2a4-44cd-87c9-c2aa1f39edc0 in namespace persistent-local-volumes-test-276 [AfterEach] [Volume type: dir-link] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:48:28.439: INFO: Deleting PersistentVolumeClaim "pvc-qptpw" May 15 01:48:28.444: INFO: Deleting PersistentVolume "local-pvn2wh2" STEP: Removing the test directory May 15 01:48:28.448: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-f56d2ae0-da77-4125-a06b-cae0ee778e1c && rm -r /tmp/local-volume-test-f56d2ae0-da77-4125-a06b-cae0ee778e1c-backend] Namespace:persistent-local-volumes-test-276 PodName:hostexec-node1-5fw6h ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:48:28.448: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "persistent-local-volumes-test-276". STEP: Found 13 events. May 15 01:48:28.586: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for hostexec-node1-5fw6h: { } Scheduled: Successfully assigned persistent-local-volumes-test-276/hostexec-node1-5fw6h to node1 May 15 01:48:28.586: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-604ae13e-a2a4-44cd-87c9-c2aa1f39edc0: { } Scheduled: Successfully assigned persistent-local-volumes-test-276/pod-604ae13e-a2a4-44cd-87c9-c2aa1f39edc0 to node1 May 15 01:48:28.586: INFO: At 2021-05-15 01:46:23 +0000 UTC - event for hostexec-node1-5fw6h: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 15 01:48:28.586: INFO: At 2021-05-15 01:46:24 +0000 UTC - event for hostexec-node1-5fw6h: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 1.295755234s May 15 01:48:28.586: INFO: At 2021-05-15 01:46:24 +0000 UTC - event for hostexec-node1-5fw6h: {kubelet node1} Created: Created container agnhost-container May 15 01:48:28.586: INFO: At 2021-05-15 01:46:24 +0000 UTC - event for hostexec-node1-5fw6h: {kubelet node1} Started: Started container agnhost-container May 15 01:48:28.586: INFO: At 2021-05-15 01:46:26 +0000 UTC - event for pvc-qptpw: {persistentvolume-controller } ProvisioningFailed: no volume plugin matched name: kubernetes.io/no-provisioner May 15 01:48:28.586: INFO: At 2021-05-15 01:46:30 +0000 UTC - event for pod-604ae13e-a2a4-44cd-87c9-c2aa1f39edc0: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:48:28.586: INFO: At 2021-05-15 01:46:30 +0000 UTC - event for pod-604ae13e-a2a4-44cd-87c9-c2aa1f39edc0: {multus } AddedInterface: Add eth0 [10.244.3.141/24] May 15 01:48:28.586: INFO: At 2021-05-15 01:46:31 +0000 UTC - event for pod-604ae13e-a2a4-44cd-87c9-c2aa1f39edc0: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:48:28.586: INFO: At 2021-05-15 01:46:31 +0000 UTC - event for pod-604ae13e-a2a4-44cd-87c9-c2aa1f39edc0: {kubelet node1} Failed: Error: ErrImagePull May 15 01:48:28.586: INFO: At 2021-05-15 01:46:32 +0000 UTC - event for pod-604ae13e-a2a4-44cd-87c9-c2aa1f39edc0: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 15 01:48:28.586: INFO: At 2021-05-15 01:46:32 +0000 UTC - event for pod-604ae13e-a2a4-44cd-87c9-c2aa1f39edc0: {kubelet node1} Failed: Error: ImagePullBackOff May 15 01:48:28.588: INFO: POD NODE PHASE GRACE CONDITIONS May 15 01:48:28.588: INFO: hostexec-node1-5fw6h node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:22 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:26 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:26 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:22 +0000 UTC }] May 15 01:48:28.588: INFO: pod-604ae13e-a2a4-44cd-87c9-c2aa1f39edc0 node1 Pending 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:28 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:28 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:28 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:28 +0000 UTC }] May 15 01:48:28.588: INFO: May 15 01:48:28.593: INFO: Logging node info for node master1 May 15 01:48:28.595: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 156373 0 2021-05-14 19:56:35 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:28 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:28 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:28 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:48:28 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[@ :],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:48:28.596: INFO: Logging kubelet events for node master1 May 15 01:48:28.600: INFO: Logging pods the kubelet thinks is on node master1 May 15 01:48:28.609: INFO: kubernetes-metrics-scraper-678c97765c-fswrn started at 2021-05-15 00:18:49 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.609: INFO: Container kubernetes-metrics-scraper ready: true, restart count 0 May 15 01:48:28.609: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.609: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:48:28.609: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.609: INFO: Container kube-multus ready: true, restart count 1 May 15 01:48:28.609: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 15 01:48:28.609: INFO: Container docker-registry ready: true, restart count 0 May 15 01:48:28.609: INFO: Container nginx ready: true, restart count 0 May 15 01:48:28.609: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.609: INFO: Container nfd-controller ready: true, restart count 0 May 15 01:48:28.609: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.609: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:48:28.609: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:48:28.609: INFO: Init container install-cni ready: true, restart count 0 May 15 01:48:28.609: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:48:28.609: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:48:28.609: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:48:28.609: INFO: Container node-exporter ready: true, restart count 0 May 15 01:48:28.609: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.609: INFO: Container kube-scheduler ready: true, restart count 0 May 15 01:48:28.609: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.609: INFO: Container kube-controller-manager ready: true, restart count 2 W0515 01:48:28.622341 32 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:48:28.653: INFO: Latency metrics for node master1 May 15 01:48:28.653: INFO: Logging node info for node master2 May 15 01:48:28.655: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 156337 0 2021-05-14 19:57:04 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:26 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:26 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:26 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:48:26 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:48:28.656: INFO: Logging kubelet events for node master2 May 15 01:48:28.661: INFO: Logging pods the kubelet thinks is on node master2 May 15 01:48:28.671: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.671: INFO: Container kube-scheduler ready: true, restart count 2 May 15 01:48:28.671: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:48:28.671: INFO: Init container install-cni ready: true, restart count 0 May 15 01:48:28.671: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:48:28.671: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.671: INFO: Container autoscaler ready: true, restart count 2 May 15 01:48:28.671: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:48:28.671: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:48:28.671: INFO: Container node-exporter ready: true, restart count 0 May 15 01:48:28.671: INFO: prometheus-operator-5bb8cb9d8f-fqb87 started at 2021-05-15 00:18:49 +0000 UTC (0+2 container statuses recorded) May 15 01:48:28.671: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:48:28.671: INFO: Container prometheus-operator ready: true, restart count 0 May 15 01:48:28.671: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.671: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:48:28.671: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.671: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:48:28.671: INFO: coredns-7677f9bb54-96w24 started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.671: INFO: Container coredns ready: true, restart count 0 May 15 01:48:28.671: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.671: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:48:28.671: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.671: INFO: Container kube-multus ready: true, restart count 1 W0515 01:48:28.684679 32 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:48:28.710: INFO: Latency metrics for node master2 May 15 01:48:28.710: INFO: Logging node info for node master3 May 15 01:48:28.713: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 156329 0 2021-05-14 19:57:14 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234767872 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324603904 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:24 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:24 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:24 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:48:24 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:48:28.713: INFO: Logging kubelet events for node master3 May 15 01:48:28.718: INFO: Logging pods the kubelet thinks is on node master3 May 15 01:48:28.725: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:48:28.725: INFO: Init container install-cni ready: true, restart count 0 May 15 01:48:28.725: INFO: Container kube-flannel ready: true, restart count 2 May 15 01:48:28.725: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.725: INFO: Container kube-multus ready: true, restart count 1 May 15 01:48:28.725: INFO: coredns-7677f9bb54-rpj8c started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.725: INFO: Container coredns ready: true, restart count 0 May 15 01:48:28.725: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:48:28.725: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:48:28.725: INFO: Container node-exporter ready: true, restart count 0 May 15 01:48:28.725: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.725: INFO: Container kube-controller-manager ready: true, restart count 3 May 15 01:48:28.725: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.725: INFO: Container kube-scheduler ready: true, restart count 3 May 15 01:48:28.725: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.725: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:48:28.725: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.725: INFO: Container kube-proxy ready: true, restart count 1 W0515 01:48:28.738064 32 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:48:28.762: INFO: Latency metrics for node master3 May 15 01:48:28.762: INFO: Logging node info for node node1 May 15 01:48:28.764: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 156328 0 2021-05-14 19:58:21 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-7734":"csi-mock-csi-mock-volumes-7734","csi-mock-csi-mock-volumes-9728":"csi-mock-csi-mock-volumes-9728","csi-mock-csi-mock-volumes-9809":"csi-mock-csi-mock-volumes-9809"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:00 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:46:49 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:47:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:24 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:24 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:24 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:48:24 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[@ :],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:48:28.765: INFO: Logging kubelet events for node node1 May 15 01:48:28.769: INFO: Logging pods the kubelet thinks is on node node1 May 15 01:48:28.787: INFO: csi-mockplugin-resizer-0 started at 2021-05-15 01:47:22 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.787: INFO: Container csi-resizer ready: true, restart count 0 May 15 01:48:28.787: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.787: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:48:28.787: INFO: pod-4cdc198f-2eab-46aa-b552-61f5a2bf0de0 started at 2021-05-15 01:46:33 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.787: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:28.787: INFO: pod-a44304e0-d8e2-42b6-8188-544b7d09e42b started at 2021-05-15 01:47:14 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.787: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:28.787: INFO: hostexec-node1-b5m77 started at 2021-05-15 01:48:00 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.787: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:28.787: INFO: pod-af39b1d5-9c8a-4ff9-938d-ae81bce79a36 started at 2021-05-15 01:48:28 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.787: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:28.787: INFO: hostexec-node1-sfsdm started at 2021-05-15 01:47:47 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.787: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:28.787: INFO: hostexec-node1-sv6vb started at 2021-05-15 01:46:26 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.787: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:28.787: INFO: pod-bec58502-0df0-4903-a0f2-b7940a229fd0 started at 2021-05-15 01:47:58 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.787: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:28.787: INFO: pod-604ae13e-a2a4-44cd-87c9-c2aa1f39edc0 started at 2021-05-15 01:46:28 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.787: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:28.787: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:48:28.787: INFO: Init container install-cni ready: true, restart count 0 May 15 01:48:28.787: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:48:28.787: INFO: node-feature-discovery-worker-bw8zg started at 2021-05-15 00:18:56 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.787: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:48:28.787: INFO: collectd-mrzps started at 2021-05-15 00:19:22 +0000 UTC (0+3 container statuses recorded) May 15 01:48:28.787: INFO: Container collectd ready: true, restart count 0 May 15 01:48:28.787: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:48:28.787: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:48:28.787: INFO: hostexec-node1-5fw6h started at 2021-05-15 01:46:22 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.787: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:28.787: INFO: hostexec-node1-5nkl7 started at 2021-05-15 01:46:52 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.787: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:28.787: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.787: INFO: Container kube-multus ready: true, restart count 1 May 15 01:48:28.787: INFO: prometheus-k8s-0 started at 2021-05-15 00:19:01 +0000 UTC (0+5 container statuses recorded) May 15 01:48:28.787: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 15 01:48:28.787: INFO: Container grafana ready: true, restart count 0 May 15 01:48:28.787: INFO: Container prometheus ready: true, restart count 26 May 15 01:48:28.787: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 15 01:48:28.787: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 15 01:48:28.787: INFO: csi-mockplugin-0 started at 2021-05-15 01:47:22 +0000 UTC (0+3 container statuses recorded) May 15 01:48:28.787: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:48:28.787: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:48:28.787: INFO: Container mock ready: true, restart count 0 May 15 01:48:28.787: INFO: hostexec-node1-xz74m started at 2021-05-15 01:47:43 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.787: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:28.787: INFO: pod-6c919e34-2cba-46c9-86ed-54b69dfbc370 started at 2021-05-15 01:47:59 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.787: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:28.787: INFO: pod-c3357cc4-fc25-4578-af0f-b315e3d2cb0c started at 2021-05-15 01:48:14 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.787: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:28.787: INFO: node-exporter-flvqz started at 2021-05-15 00:18:55 +0000 UTC (0+2 container statuses recorded) May 15 01:48:28.787: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:48:28.787: INFO: Container node-exporter ready: true, restart count 0 May 15 01:48:28.787: INFO: pvc-volume-tester-hwvvq started at 2021-05-15 01:47:34 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.787: INFO: Container volume-tester ready: true, restart count 0 May 15 01:48:28.787: INFO: hostexec-node1-r82s6 started at 2021-05-15 01:46:27 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.787: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:28.787: INFO: cmk-4s6dm started at 2021-05-15 00:18:54 +0000 UTC (0+2 container statuses recorded) May 15 01:48:28.787: INFO: Container nodereport ready: true, restart count 0 May 15 01:48:28.787: INFO: Container reconcile ready: true, restart count 0 May 15 01:48:28.787: INFO: hostexec-node1-wmdkc started at 2021-05-15 01:48:21 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.787: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:28.787: INFO: pod-470e75a2-4056-4cd4-838e-f67be7033ced started at 2021-05-15 01:46:43 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.787: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:28.787: INFO: pod-dbfe8699-c076-46f1-bf35-86df11b4addd started at 2021-05-15 01:46:42 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.787: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:28.787: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.787: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:48:28.787: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4vxcc started at 2021-05-15 00:19:00 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.787: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:48:28.787: INFO: hostexec-node1-frmv7 started at 2021-05-15 01:46:31 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.787: INFO: Container agnhost-container ready: true, restart count 0 W0515 01:48:28.800432 32 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:48:28.909: INFO: Latency metrics for node node1 May 15 01:48:28.909: INFO: Logging node info for node node2 May 15 01:48:28.912: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 156280 0 2021-05-14 19:58:22 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-3052":"csi-mock-csi-mock-volumes-3052","csi-mock-csi-mock-volumes-8760":"csi-mock-csi-mock-volumes-8760","csi-mock-csi-mock-volumes-9624":"csi-mock-csi-mock-volumes-9624"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:46 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:44:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:46:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:io.kubernetes.storage.mock/node":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:19 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:19 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:19 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:48:19 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:48:28.913: INFO: Logging kubelet events for node node2 May 15 01:48:28.917: INFO: Logging pods the kubelet thinks is on node node2 May 15 01:48:28.929: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 15 01:48:28.929: INFO: Container discover ready: false, restart count 0 May 15 01:48:28.929: INFO: Container init ready: false, restart count 0 May 15 01:48:28.929: INFO: Container install ready: false, restart count 0 May 15 01:48:28.929: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.929: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:48:28.929: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.929: INFO: Container kube-multus ready: true, restart count 1 May 15 01:48:28.929: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.929: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 15 01:48:28.929: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.929: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:48:28.929: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:48:28.929: INFO: Init container install-cni ready: true, restart count 1 May 15 01:48:28.929: INFO: Container kube-flannel ready: true, restart count 4 May 15 01:48:28.929: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:48:28.929: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:48:28.929: INFO: Container node-exporter ready: true, restart count 0 May 15 01:48:28.929: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 15 01:48:28.929: INFO: Container tas-controller ready: true, restart count 0 May 15 01:48:28.929: INFO: Container tas-extender ready: true, restart count 0 May 15 01:48:28.929: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 15 01:48:28.929: INFO: Container collectd ready: true, restart count 0 May 15 01:48:28.929: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:48:28.929: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:48:28.929: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.929: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:48:28.929: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.929: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:48:28.929: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 15 01:48:28.929: INFO: Container nodereport ready: true, restart count 0 May 15 01:48:28.929: INFO: Container reconcile ready: true, restart count 0 May 15 01:48:28.929: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 15 01:48:28.929: INFO: Container cmk-webhook ready: true, restart count 0 W0515 01:48:28.942459 32 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:48:28.975: INFO: Latency metrics for node node2 May 15 01:48:28.975: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-276" for this suite. • Failure in Spec Setup (BeforeEach) [126.752 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: dir-link] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:205 should be able to mount volume and write from pod1 [BeforeEach] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:234 May 15 01:48:28.433: Unexpected error: <*errors.errorString | 0xc0009708b0>: { s: "pod \"pod-604ae13e-a2a4-44cd-87c9-c2aa1f39edc0\" is not Running: timed out waiting for the condition", } pod "pod-604ae13e-a2a4-44cd-87c9-c2aa1f39edc0" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 ------------------------------ {"msg":"FAILED [sig-storage] PersistentVolumes-local [Volume type: dir-link] One pod requesting one prebound PVC should be able to mount volume and write from pod1","total":-1,"completed":1,"skipped":54,"failed":2,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir-link] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir-link] One pod requesting one prebound PVC should be able to mount volume and write from pod1"]} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:46:27.304: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: blockfswithoutformat] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes STEP: Creating block device on node "node1" using path "/tmp/local-volume-test-f9595832-2835-4c91-ac18-5f2470fec065" May 15 01:46:31.348: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-f9595832-2835-4c91-ac18-5f2470fec065 && dd if=/dev/zero of=/tmp/local-volume-test-f9595832-2835-4c91-ac18-5f2470fec065/file bs=4096 count=5120 && losetup -f /tmp/local-volume-test-f9595832-2835-4c91-ac18-5f2470fec065/file] Namespace:persistent-local-volumes-test-2101 PodName:hostexec-node1-r82s6 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:46:31.348: INFO: >>> kubeConfig: /root/.kube/config May 15 01:46:31.533: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c E2E_LOOP_DEV=$(losetup | grep /tmp/local-volume-test-f9595832-2835-4c91-ac18-5f2470fec065/file | awk '{ print $1 }') 2>&1 > /dev/null && echo ${E2E_LOOP_DEV}] Namespace:persistent-local-volumes-test-2101 PodName:hostexec-node1-r82s6 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:46:31.533: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:46:31.671: INFO: Creating a PV followed by a PVC May 15 01:46:31.677: INFO: Waiting for PV local-pvzkwt7 to bind to PVC pvc-hsddg May 15 01:46:31.677: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-hsddg] to have phase Bound May 15 01:46:31.679: INFO: PersistentVolumeClaim pvc-hsddg found but phase is Pending instead of Bound. May 15 01:46:33.683: INFO: PersistentVolumeClaim pvc-hsddg found and phase=Bound (2.006357238s) May 15 01:46:33.684: INFO: Waiting up to 3m0s for PersistentVolume local-pvzkwt7 to have phase Bound May 15 01:46:33.687: INFO: PersistentVolume local-pvzkwt7 found and phase=Bound (3.912092ms) [BeforeEach] One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:211 STEP: Creating pod1 STEP: Creating a pod May 15 01:48:33.712: FAIL: Unexpected error: <*errors.errorString | 0xc000750660>: { s: "pod \"pod-4cdc198f-2eab-46aa-b552-61f5a2bf0de0\" is not Running: timed out waiting for the condition", } pod "pod-4cdc198f-2eab-46aa-b552-61f5a2bf0de0" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage.glob..func20.2.3.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 +0x119 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc0026cca80) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc0026cca80) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc0026cca80, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:223 STEP: Deleting pod1 STEP: Deleting pod pod-4cdc198f-2eab-46aa-b552-61f5a2bf0de0 in namespace persistent-local-volumes-test-2101 [AfterEach] [Volume type: blockfswithoutformat] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:48:33.718: INFO: Deleting PersistentVolumeClaim "pvc-hsddg" May 15 01:48:33.722: INFO: Deleting PersistentVolume "local-pvzkwt7" May 15 01:48:33.725: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c E2E_LOOP_DEV=$(losetup | grep /tmp/local-volume-test-f9595832-2835-4c91-ac18-5f2470fec065/file | awk '{ print $1 }') 2>&1 > /dev/null && echo ${E2E_LOOP_DEV}] Namespace:persistent-local-volumes-test-2101 PodName:hostexec-node1-r82s6 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:48:33.725: INFO: >>> kubeConfig: /root/.kube/config STEP: Tear down block device "/dev/loop0" on node "node1" at path /tmp/local-volume-test-f9595832-2835-4c91-ac18-5f2470fec065/file May 15 01:48:33.964: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c losetup -d /dev/loop0] Namespace:persistent-local-volumes-test-2101 PodName:hostexec-node1-r82s6 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:48:33.964: INFO: >>> kubeConfig: /root/.kube/config STEP: Removing the test directory /tmp/local-volume-test-f9595832-2835-4c91-ac18-5f2470fec065 May 15 01:48:34.109: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-f9595832-2835-4c91-ac18-5f2470fec065] Namespace:persistent-local-volumes-test-2101 PodName:hostexec-node1-r82s6 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:48:34.110: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "persistent-local-volumes-test-2101". STEP: Found 12 events. May 15 01:48:34.273: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for hostexec-node1-r82s6: { } Scheduled: Successfully assigned persistent-local-volumes-test-2101/hostexec-node1-r82s6 to node1 May 15 01:48:34.273: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-4cdc198f-2eab-46aa-b552-61f5a2bf0de0: { } Scheduled: Successfully assigned persistent-local-volumes-test-2101/pod-4cdc198f-2eab-46aa-b552-61f5a2bf0de0 to node1 May 15 01:48:34.273: INFO: At 2021-05-15 01:46:28 +0000 UTC - event for hostexec-node1-r82s6: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 15 01:48:34.273: INFO: At 2021-05-15 01:46:28 +0000 UTC - event for hostexec-node1-r82s6: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 473.877502ms May 15 01:48:34.273: INFO: At 2021-05-15 01:46:29 +0000 UTC - event for hostexec-node1-r82s6: {kubelet node1} Created: Created container agnhost-container May 15 01:48:34.273: INFO: At 2021-05-15 01:46:29 +0000 UTC - event for hostexec-node1-r82s6: {kubelet node1} Started: Started container agnhost-container May 15 01:48:34.273: INFO: At 2021-05-15 01:46:38 +0000 UTC - event for pod-4cdc198f-2eab-46aa-b552-61f5a2bf0de0: {multus } AddedInterface: Add eth0 [10.244.3.146/24] May 15 01:48:34.273: INFO: At 2021-05-15 01:46:38 +0000 UTC - event for pod-4cdc198f-2eab-46aa-b552-61f5a2bf0de0: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:48:34.273: INFO: At 2021-05-15 01:46:39 +0000 UTC - event for pod-4cdc198f-2eab-46aa-b552-61f5a2bf0de0: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:48:34.273: INFO: At 2021-05-15 01:46:39 +0000 UTC - event for pod-4cdc198f-2eab-46aa-b552-61f5a2bf0de0: {kubelet node1} Failed: Error: ErrImagePull May 15 01:48:34.273: INFO: At 2021-05-15 01:46:39 +0000 UTC - event for pod-4cdc198f-2eab-46aa-b552-61f5a2bf0de0: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 15 01:48:34.273: INFO: At 2021-05-15 01:46:39 +0000 UTC - event for pod-4cdc198f-2eab-46aa-b552-61f5a2bf0de0: {kubelet node1} Failed: Error: ImagePullBackOff May 15 01:48:34.275: INFO: POD NODE PHASE GRACE CONDITIONS May 15 01:48:34.275: INFO: hostexec-node1-r82s6 node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:27 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:29 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:29 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:27 +0000 UTC }] May 15 01:48:34.276: INFO: pod-4cdc198f-2eab-46aa-b552-61f5a2bf0de0 node1 Pending 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:33 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:33 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:33 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:33 +0000 UTC }] May 15 01:48:34.276: INFO: May 15 01:48:34.279: INFO: Logging node info for node master1 May 15 01:48:34.282: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 156373 0 2021-05-14 19:56:35 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:28 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:28 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:28 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:48:28 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[@ :],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:48:34.282: INFO: Logging kubelet events for node master1 May 15 01:48:34.286: INFO: Logging pods the kubelet thinks is on node master1 May 15 01:48:34.305: INFO: kubernetes-metrics-scraper-678c97765c-fswrn started at 2021-05-15 00:18:49 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.305: INFO: Container kubernetes-metrics-scraper ready: true, restart count 0 May 15 01:48:34.305: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.305: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:48:34.305: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.305: INFO: Container kube-multus ready: true, restart count 1 May 15 01:48:34.305: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 15 01:48:34.305: INFO: Container docker-registry ready: true, restart count 0 May 15 01:48:34.305: INFO: Container nginx ready: true, restart count 0 May 15 01:48:34.305: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.305: INFO: Container nfd-controller ready: true, restart count 0 May 15 01:48:34.305: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.305: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:48:34.305: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:48:34.305: INFO: Init container install-cni ready: true, restart count 0 May 15 01:48:34.305: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:48:34.305: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:48:34.305: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:48:34.305: INFO: Container node-exporter ready: true, restart count 0 May 15 01:48:34.305: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.305: INFO: Container kube-scheduler ready: true, restart count 0 May 15 01:48:34.305: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.305: INFO: Container kube-controller-manager ready: true, restart count 2 W0515 01:48:34.317151 30 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:48:34.350: INFO: Latency metrics for node master1 May 15 01:48:34.350: INFO: Logging node info for node master2 May 15 01:48:34.352: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 156337 0 2021-05-14 19:57:04 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:26 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:26 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:26 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:48:26 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:48:34.353: INFO: Logging kubelet events for node master2 May 15 01:48:34.358: INFO: Logging pods the kubelet thinks is on node master2 May 15 01:48:34.367: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.367: INFO: Container kube-scheduler ready: true, restart count 2 May 15 01:48:34.367: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:48:34.367: INFO: Init container install-cni ready: true, restart count 0 May 15 01:48:34.367: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:48:34.367: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.367: INFO: Container autoscaler ready: true, restart count 2 May 15 01:48:34.367: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:48:34.367: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:48:34.367: INFO: Container node-exporter ready: true, restart count 0 May 15 01:48:34.367: INFO: prometheus-operator-5bb8cb9d8f-fqb87 started at 2021-05-15 00:18:49 +0000 UTC (0+2 container statuses recorded) May 15 01:48:34.367: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:48:34.367: INFO: Container prometheus-operator ready: true, restart count 0 May 15 01:48:34.367: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.367: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:48:34.367: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.367: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:48:34.367: INFO: coredns-7677f9bb54-96w24 started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.367: INFO: Container coredns ready: true, restart count 0 May 15 01:48:34.367: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.367: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:48:34.367: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.367: INFO: Container kube-multus ready: true, restart count 1 W0515 01:48:34.379867 30 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:48:34.405: INFO: Latency metrics for node master2 May 15 01:48:34.405: INFO: Logging node info for node master3 May 15 01:48:34.407: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 156456 0 2021-05-14 19:57:14 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234767872 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324603904 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:34 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:34 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:34 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:48:34 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:48:34.408: INFO: Logging kubelet events for node master3 May 15 01:48:34.412: INFO: Logging pods the kubelet thinks is on node master3 May 15 01:48:34.419: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.419: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:48:34.419: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.419: INFO: Container kube-proxy ready: true, restart count 1 May 15 01:48:34.419: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:48:34.419: INFO: Init container install-cni ready: true, restart count 0 May 15 01:48:34.419: INFO: Container kube-flannel ready: true, restart count 2 May 15 01:48:34.419: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.419: INFO: Container kube-multus ready: true, restart count 1 May 15 01:48:34.419: INFO: coredns-7677f9bb54-rpj8c started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.419: INFO: Container coredns ready: true, restart count 0 May 15 01:48:34.419: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:48:34.419: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:48:34.419: INFO: Container node-exporter ready: true, restart count 0 May 15 01:48:34.419: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.419: INFO: Container kube-controller-manager ready: true, restart count 3 May 15 01:48:34.419: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.419: INFO: Container kube-scheduler ready: true, restart count 3 W0515 01:48:34.429856 30 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:48:34.458: INFO: Latency metrics for node master3 May 15 01:48:34.458: INFO: Logging node info for node node1 May 15 01:48:34.461: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 156458 0 2021-05-14 19:58:21 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-7734":"csi-mock-csi-mock-volumes-7734","csi-mock-csi-mock-volumes-9728":"csi-mock-csi-mock-volumes-9728","csi-mock-csi-mock-volumes-9809":"csi-mock-csi-mock-volumes-9809"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:00 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:46:49 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:47:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:34 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:34 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:34 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:48:34 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[@ :],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:48:34.461: INFO: Logging kubelet events for node node1 May 15 01:48:34.465: INFO: Logging pods the kubelet thinks is on node node1 May 15 01:48:34.484: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:48:34.484: INFO: Init container install-cni ready: true, restart count 0 May 15 01:48:34.484: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:48:34.484: INFO: node-feature-discovery-worker-bw8zg started at 2021-05-15 00:18:56 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.484: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:48:34.484: INFO: collectd-mrzps started at 2021-05-15 00:19:22 +0000 UTC (0+3 container statuses recorded) May 15 01:48:34.484: INFO: Container collectd ready: true, restart count 0 May 15 01:48:34.484: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:48:34.484: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:48:34.484: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.484: INFO: Container kube-multus ready: true, restart count 1 May 15 01:48:34.484: INFO: prometheus-k8s-0 started at 2021-05-15 00:19:01 +0000 UTC (0+5 container statuses recorded) May 15 01:48:34.484: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 15 01:48:34.484: INFO: Container grafana ready: true, restart count 0 May 15 01:48:34.484: INFO: Container prometheus ready: true, restart count 26 May 15 01:48:34.484: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 15 01:48:34.484: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 15 01:48:34.484: INFO: csi-mockplugin-0 started at 2021-05-15 01:47:22 +0000 UTC (0+3 container statuses recorded) May 15 01:48:34.484: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:48:34.484: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:48:34.484: INFO: Container mock ready: true, restart count 0 May 15 01:48:34.484: INFO: hostexec-node1-5nkl7 started at 2021-05-15 01:46:52 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.484: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:34.484: INFO: node-exporter-flvqz started at 2021-05-15 00:18:55 +0000 UTC (0+2 container statuses recorded) May 15 01:48:34.484: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:48:34.484: INFO: Container node-exporter ready: true, restart count 0 May 15 01:48:34.484: INFO: pvc-volume-tester-hwvvq started at 2021-05-15 01:47:34 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.484: INFO: Container volume-tester ready: true, restart count 0 May 15 01:48:34.484: INFO: hostexec-node1-xz74m started at 2021-05-15 01:47:43 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.484: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:34.484: INFO: pod-6c919e34-2cba-46c9-86ed-54b69dfbc370 started at 2021-05-15 01:47:59 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.484: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:34.484: INFO: pod-c3357cc4-fc25-4578-af0f-b315e3d2cb0c started at 2021-05-15 01:48:14 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.484: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:34.484: INFO: hostexec-node1-r82s6 started at 2021-05-15 01:46:27 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.484: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:34.484: INFO: cmk-4s6dm started at 2021-05-15 00:18:54 +0000 UTC (0+2 container statuses recorded) May 15 01:48:34.484: INFO: Container nodereport ready: true, restart count 0 May 15 01:48:34.484: INFO: Container reconcile ready: true, restart count 0 May 15 01:48:34.484: INFO: hostexec-node1-wmdkc started at 2021-05-15 01:48:21 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.484: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:34.484: INFO: pod-470e75a2-4056-4cd4-838e-f67be7033ced started at 2021-05-15 01:46:43 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.484: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:34.484: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.484: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:48:34.484: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4vxcc started at 2021-05-15 00:19:00 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.484: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:48:34.484: INFO: hostexec-node1-frmv7 started at 2021-05-15 01:46:31 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.484: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:34.484: INFO: pod-dbfe8699-c076-46f1-bf35-86df11b4addd started at 2021-05-15 01:46:42 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.484: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:34.484: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.484: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:48:34.484: INFO: pod-4cdc198f-2eab-46aa-b552-61f5a2bf0de0 started at 2021-05-15 01:46:33 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.484: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:34.484: INFO: pod-a44304e0-d8e2-42b6-8188-544b7d09e42b started at 2021-05-15 01:47:14 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.484: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:34.484: INFO: csi-mockplugin-resizer-0 started at 2021-05-15 01:47:22 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.484: INFO: Container csi-resizer ready: true, restart count 0 May 15 01:48:34.484: INFO: hostexec-node1-sfsdm started at 2021-05-15 01:47:47 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.484: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:34.484: INFO: hostexec-node1-sv6vb started at 2021-05-15 01:46:26 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.484: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:34.484: INFO: pod-bec58502-0df0-4903-a0f2-b7940a229fd0 started at 2021-05-15 01:47:58 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.484: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:34.484: INFO: hostexec-node1-b5m77 started at 2021-05-15 01:48:00 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.484: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:34.484: INFO: pod-af39b1d5-9c8a-4ff9-938d-ae81bce79a36 started at 2021-05-15 01:48:28 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.484: INFO: Container write-pod ready: false, restart count 0 W0515 01:48:34.497857 30 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:48:34.551: INFO: Latency metrics for node node1 May 15 01:48:34.551: INFO: Logging node info for node node2 May 15 01:48:34.554: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 156397 0 2021-05-14 19:58:22 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-3052":"csi-mock-csi-mock-volumes-3052","csi-mock-csi-mock-volumes-8760":"csi-mock-csi-mock-volumes-8760","csi-mock-csi-mock-volumes-9624":"csi-mock-csi-mock-volumes-9624"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:46 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:44:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:46:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:io.kubernetes.storage.mock/node":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:29 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:29 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:29 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:48:29 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:48:34.555: INFO: Logging kubelet events for node node2 May 15 01:48:34.559: INFO: Logging pods the kubelet thinks is on node node2 May 15 01:48:34.570: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.570: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:48:34.570: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:48:34.571: INFO: Init container install-cni ready: true, restart count 1 May 15 01:48:34.571: INFO: Container kube-flannel ready: true, restart count 4 May 15 01:48:34.571: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:48:34.571: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:48:34.571: INFO: Container node-exporter ready: true, restart count 0 May 15 01:48:34.571: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 15 01:48:34.571: INFO: Container tas-controller ready: true, restart count 0 May 15 01:48:34.571: INFO: Container tas-extender ready: true, restart count 0 May 15 01:48:34.571: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 15 01:48:34.571: INFO: Container collectd ready: true, restart count 0 May 15 01:48:34.571: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:48:34.571: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:48:34.571: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.571: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:48:34.571: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.571: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:48:34.571: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 15 01:48:34.571: INFO: Container nodereport ready: true, restart count 0 May 15 01:48:34.571: INFO: Container reconcile ready: true, restart count 0 May 15 01:48:34.571: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.571: INFO: Container cmk-webhook ready: true, restart count 0 May 15 01:48:34.571: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 15 01:48:34.571: INFO: Container discover ready: false, restart count 0 May 15 01:48:34.571: INFO: Container init ready: false, restart count 0 May 15 01:48:34.571: INFO: Container install ready: false, restart count 0 May 15 01:48:34.571: INFO: pod-ephm-test-projected-8bh5 started at 2021-05-15 01:48:29 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.571: INFO: Container test-container-subpath-projected-8bh5 ready: false, restart count 0 May 15 01:48:34.571: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.571: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:48:34.571: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.571: INFO: Container kube-multus ready: true, restart count 1 May 15 01:48:34.571: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 15 01:48:34.571: INFO: Container kubernetes-dashboard ready: true, restart count 2 W0515 01:48:34.586242 30 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:48:34.624: INFO: Latency metrics for node node2 May 15 01:48:34.624: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-2101" for this suite. • Failure in Spec Setup (BeforeEach) [127.329 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: blockfswithoutformat] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:205 should be able to mount volume and write from pod1 [BeforeEach] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:234 May 15 01:48:33.712: Unexpected error: <*errors.errorString | 0xc000750660>: { s: "pod \"pod-4cdc198f-2eab-46aa-b552-61f5a2bf0de0\" is not Running: timed out waiting for the condition", } pod "pod-4cdc198f-2eab-46aa-b552-61f5a2bf0de0" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 ------------------------------ {"msg":"FAILED [sig-storage] PersistentVolumes-local [Volume type: blockfswithoutformat] One pod requesting one prebound PVC should be able to mount volume and write from pod1","total":-1,"completed":2,"skipped":39,"failed":1,"failures":["[sig-storage] PersistentVolumes-local [Volume type: blockfswithoutformat] One pod requesting one prebound PVC should be able to mount volume and write from pod1"]} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:46:26.744: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: tmpfs] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes STEP: Creating tmpfs mount point on node "node1" at path "/tmp/local-volume-test-c301f3b7-ec62-4f93-8faa-5dedb7663051" May 15 01:46:30.794: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p "/tmp/local-volume-test-c301f3b7-ec62-4f93-8faa-5dedb7663051" && mount -t tmpfs -o size=10m tmpfs-"/tmp/local-volume-test-c301f3b7-ec62-4f93-8faa-5dedb7663051" "/tmp/local-volume-test-c301f3b7-ec62-4f93-8faa-5dedb7663051"] Namespace:persistent-local-volumes-test-5862 PodName:hostexec-node1-sv6vb ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:46:30.794: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:46:30.932: INFO: Creating a PV followed by a PVC May 15 01:46:30.939: INFO: Waiting for PV local-pvb7nf2 to bind to PVC pvc-x7q4l May 15 01:46:30.939: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-x7q4l] to have phase Bound May 15 01:46:30.941: INFO: PersistentVolumeClaim pvc-x7q4l found but phase is Pending instead of Bound. May 15 01:46:32.945: INFO: PersistentVolumeClaim pvc-x7q4l found but phase is Pending instead of Bound. May 15 01:46:34.952: INFO: PersistentVolumeClaim pvc-x7q4l found but phase is Pending instead of Bound. May 15 01:46:36.956: INFO: PersistentVolumeClaim pvc-x7q4l found but phase is Pending instead of Bound. May 15 01:46:38.959: INFO: PersistentVolumeClaim pvc-x7q4l found but phase is Pending instead of Bound. May 15 01:46:40.962: INFO: PersistentVolumeClaim pvc-x7q4l found but phase is Pending instead of Bound. May 15 01:46:42.965: INFO: PersistentVolumeClaim pvc-x7q4l found and phase=Bound (12.02570052s) May 15 01:46:42.965: INFO: Waiting up to 3m0s for PersistentVolume local-pvb7nf2 to have phase Bound May 15 01:46:42.967: INFO: PersistentVolume local-pvb7nf2 found and phase=Bound (1.909626ms) [BeforeEach] One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:211 STEP: Creating pod1 STEP: Creating a pod May 15 01:48:42.991: FAIL: Unexpected error: <*errors.errorString | 0xc0048a4480>: { s: "pod \"pod-dbfe8699-c076-46f1-bf35-86df11b4addd\" is not Running: timed out waiting for the condition", } pod "pod-dbfe8699-c076-46f1-bf35-86df11b4addd" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage.glob..func20.2.3.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 +0x119 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc001881380) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc001881380) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc001881380, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:223 STEP: Deleting pod1 STEP: Deleting pod pod-dbfe8699-c076-46f1-bf35-86df11b4addd in namespace persistent-local-volumes-test-5862 [AfterEach] [Volume type: tmpfs] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:48:42.999: INFO: Deleting PersistentVolumeClaim "pvc-x7q4l" May 15 01:48:43.002: INFO: Deleting PersistentVolume "local-pvb7nf2" STEP: Unmount tmpfs mount point on node "node1" at path "/tmp/local-volume-test-c301f3b7-ec62-4f93-8faa-5dedb7663051" May 15 01:48:43.006: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c umount "/tmp/local-volume-test-c301f3b7-ec62-4f93-8faa-5dedb7663051"] Namespace:persistent-local-volumes-test-5862 PodName:hostexec-node1-sv6vb ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:48:43.006: INFO: >>> kubeConfig: /root/.kube/config STEP: Removing the test directory May 15 01:48:43.153: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-c301f3b7-ec62-4f93-8faa-5dedb7663051] Namespace:persistent-local-volumes-test-5862 PodName:hostexec-node1-sv6vb ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:48:43.153: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "persistent-local-volumes-test-5862". STEP: Found 15 events. May 15 01:48:43.272: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for hostexec-node1-sv6vb: { } Scheduled: Successfully assigned persistent-local-volumes-test-5862/hostexec-node1-sv6vb to node1 May 15 01:48:43.272: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-dbfe8699-c076-46f1-bf35-86df11b4addd: { } Scheduled: Successfully assigned persistent-local-volumes-test-5862/pod-dbfe8699-c076-46f1-bf35-86df11b4addd to node1 May 15 01:48:43.272: INFO: At 2021-05-15 01:46:27 +0000 UTC - event for hostexec-node1-sv6vb: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 15 01:48:43.272: INFO: At 2021-05-15 01:46:27 +0000 UTC - event for hostexec-node1-sv6vb: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 469.949851ms May 15 01:48:43.272: INFO: At 2021-05-15 01:46:28 +0000 UTC - event for hostexec-node1-sv6vb: {kubelet node1} Created: Created container agnhost-container May 15 01:48:43.272: INFO: At 2021-05-15 01:46:28 +0000 UTC - event for hostexec-node1-sv6vb: {kubelet node1} Started: Started container agnhost-container May 15 01:48:43.272: INFO: At 2021-05-15 01:46:30 +0000 UTC - event for pvc-x7q4l: {persistentvolume-controller } ProvisioningFailed: no volume plugin matched name: kubernetes.io/no-provisioner May 15 01:48:43.272: INFO: At 2021-05-15 01:46:46 +0000 UTC - event for pod-dbfe8699-c076-46f1-bf35-86df11b4addd: {multus } AddedInterface: Add eth0 [10.244.3.147/24] May 15 01:48:43.272: INFO: At 2021-05-15 01:46:46 +0000 UTC - event for pod-dbfe8699-c076-46f1-bf35-86df11b4addd: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:48:43.272: INFO: At 2021-05-15 01:46:47 +0000 UTC - event for pod-dbfe8699-c076-46f1-bf35-86df11b4addd: {kubelet node1} Failed: Error: ErrImagePull May 15 01:48:43.272: INFO: At 2021-05-15 01:46:47 +0000 UTC - event for pod-dbfe8699-c076-46f1-bf35-86df11b4addd: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:48:43.272: INFO: At 2021-05-15 01:46:48 +0000 UTC - event for pod-dbfe8699-c076-46f1-bf35-86df11b4addd: {kubelet node1} SandboxChanged: Pod sandbox changed, it will be killed and re-created. May 15 01:48:43.272: INFO: At 2021-05-15 01:46:50 +0000 UTC - event for pod-dbfe8699-c076-46f1-bf35-86df11b4addd: {multus } AddedInterface: Add eth0 [10.244.3.149/24] May 15 01:48:43.272: INFO: At 2021-05-15 01:46:50 +0000 UTC - event for pod-dbfe8699-c076-46f1-bf35-86df11b4addd: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 15 01:48:43.272: INFO: At 2021-05-15 01:46:50 +0000 UTC - event for pod-dbfe8699-c076-46f1-bf35-86df11b4addd: {kubelet node1} Failed: Error: ImagePullBackOff May 15 01:48:43.276: INFO: POD NODE PHASE GRACE CONDITIONS May 15 01:48:43.276: INFO: hostexec-node1-sv6vb node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:26 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:29 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:29 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:26 +0000 UTC }] May 15 01:48:43.276: INFO: pod-dbfe8699-c076-46f1-bf35-86df11b4addd node1 Pending 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:42 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:42 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:42 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:42 +0000 UTC }] May 15 01:48:43.277: INFO: May 15 01:48:43.281: INFO: Logging node info for node master1 May 15 01:48:43.284: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 156512 0 2021-05-14 19:56:35 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:38 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:38 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:38 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:48:38 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[@ :],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:48:43.284: INFO: Logging kubelet events for node master1 May 15 01:48:43.288: INFO: Logging pods the kubelet thinks is on node master1 May 15 01:48:43.297: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.297: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:48:43.297: INFO: kubernetes-metrics-scraper-678c97765c-fswrn started at 2021-05-15 00:18:49 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.297: INFO: Container kubernetes-metrics-scraper ready: true, restart count 0 May 15 01:48:43.297: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.297: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:48:43.297: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.297: INFO: Container kube-multus ready: true, restart count 1 May 15 01:48:43.297: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 15 01:48:43.297: INFO: Container docker-registry ready: true, restart count 0 May 15 01:48:43.297: INFO: Container nginx ready: true, restart count 0 May 15 01:48:43.297: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.297: INFO: Container nfd-controller ready: true, restart count 0 May 15 01:48:43.297: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.297: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:48:43.297: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:48:43.297: INFO: Init container install-cni ready: true, restart count 0 May 15 01:48:43.297: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:48:43.297: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:48:43.297: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:48:43.297: INFO: Container node-exporter ready: true, restart count 0 May 15 01:48:43.297: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.297: INFO: Container kube-scheduler ready: true, restart count 0 W0515 01:48:43.308498 24 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:48:43.341: INFO: Latency metrics for node master1 May 15 01:48:43.341: INFO: Logging node info for node master2 May 15 01:48:43.343: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 156493 0 2021-05-14 19:57:04 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:36 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:36 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:36 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:48:36 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:48:43.343: INFO: Logging kubelet events for node master2 May 15 01:48:43.348: INFO: Logging pods the kubelet thinks is on node master2 May 15 01:48:43.357: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.357: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:48:43.357: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.357: INFO: Container kube-multus ready: true, restart count 1 May 15 01:48:43.357: INFO: coredns-7677f9bb54-96w24 started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.357: INFO: Container coredns ready: true, restart count 0 May 15 01:48:43.357: INFO: prometheus-operator-5bb8cb9d8f-fqb87 started at 2021-05-15 00:18:49 +0000 UTC (0+2 container statuses recorded) May 15 01:48:43.357: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:48:43.357: INFO: Container prometheus-operator ready: true, restart count 0 May 15 01:48:43.357: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.357: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:48:43.357: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.357: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:48:43.357: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.357: INFO: Container kube-scheduler ready: true, restart count 2 May 15 01:48:43.357: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:48:43.357: INFO: Init container install-cni ready: true, restart count 0 May 15 01:48:43.357: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:48:43.357: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.357: INFO: Container autoscaler ready: true, restart count 2 May 15 01:48:43.357: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:48:43.357: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:48:43.357: INFO: Container node-exporter ready: true, restart count 0 W0515 01:48:43.371120 24 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:48:43.396: INFO: Latency metrics for node master2 May 15 01:48:43.396: INFO: Logging node info for node master3 May 15 01:48:43.398: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 156456 0 2021-05-14 19:57:14 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234767872 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324603904 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:34 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:34 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:34 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:48:34 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:48:43.399: INFO: Logging kubelet events for node master3 May 15 01:48:43.403: INFO: Logging pods the kubelet thinks is on node master3 May 15 01:48:43.410: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.410: INFO: Container kube-scheduler ready: true, restart count 3 May 15 01:48:43.410: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.410: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:48:43.410: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.410: INFO: Container kube-proxy ready: true, restart count 1 May 15 01:48:43.410: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:48:43.410: INFO: Init container install-cni ready: true, restart count 0 May 15 01:48:43.410: INFO: Container kube-flannel ready: true, restart count 2 May 15 01:48:43.410: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.410: INFO: Container kube-multus ready: true, restart count 1 May 15 01:48:43.410: INFO: coredns-7677f9bb54-rpj8c started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.410: INFO: Container coredns ready: true, restart count 0 May 15 01:48:43.410: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:48:43.410: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:48:43.410: INFO: Container node-exporter ready: true, restart count 0 May 15 01:48:43.410: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.410: INFO: Container kube-controller-manager ready: true, restart count 3 W0515 01:48:43.422305 24 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:48:43.450: INFO: Latency metrics for node master3 May 15 01:48:43.450: INFO: Logging node info for node node1 May 15 01:48:43.453: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 156458 0 2021-05-14 19:58:21 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-7734":"csi-mock-csi-mock-volumes-7734","csi-mock-csi-mock-volumes-9728":"csi-mock-csi-mock-volumes-9728","csi-mock-csi-mock-volumes-9809":"csi-mock-csi-mock-volumes-9809"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:00 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:46:49 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:47:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:34 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:34 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:34 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:48:34 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[@ :],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:48:43.453: INFO: Logging kubelet events for node node1 May 15 01:48:43.457: INFO: Logging pods the kubelet thinks is on node node1 May 15 01:48:43.474: INFO: hostexec-node1-wmdkc started at 2021-05-15 01:48:21 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.474: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:43.474: INFO: pod-470e75a2-4056-4cd4-838e-f67be7033ced started at 2021-05-15 01:46:43 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.474: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:43.474: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.474: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:48:43.474: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4vxcc started at 2021-05-15 00:19:00 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.474: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:48:43.474: INFO: hostexec-node1-frmv7 started at 2021-05-15 01:46:31 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.474: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:43.474: INFO: pod-dbfe8699-c076-46f1-bf35-86df11b4addd started at 2021-05-15 01:46:42 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.474: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:43.474: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.474: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:48:43.474: INFO: pod-a44304e0-d8e2-42b6-8188-544b7d09e42b started at 2021-05-15 01:47:14 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.474: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:43.474: INFO: csi-mockplugin-resizer-0 started at 2021-05-15 01:47:22 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.474: INFO: Container csi-resizer ready: true, restart count 0 May 15 01:48:43.474: INFO: pod-af39b1d5-9c8a-4ff9-938d-ae81bce79a36 started at 2021-05-15 01:48:28 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.474: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:43.474: INFO: hostexec-node1-sfsdm started at 2021-05-15 01:47:47 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.474: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:43.474: INFO: hostexec-node1-sv6vb started at 2021-05-15 01:46:26 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.474: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:43.474: INFO: pod-bec58502-0df0-4903-a0f2-b7940a229fd0 started at 2021-05-15 01:47:58 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.474: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:43.474: INFO: hostexec-node1-b5m77 started at 2021-05-15 01:48:00 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.474: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:43.474: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:48:43.474: INFO: Init container install-cni ready: true, restart count 0 May 15 01:48:43.474: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:48:43.474: INFO: node-feature-discovery-worker-bw8zg started at 2021-05-15 00:18:56 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.474: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:48:43.474: INFO: collectd-mrzps started at 2021-05-15 00:19:22 +0000 UTC (0+3 container statuses recorded) May 15 01:48:43.474: INFO: Container collectd ready: true, restart count 0 May 15 01:48:43.474: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:48:43.474: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:48:43.474: INFO: hostexec-node1-8qtn9 started at 2021-05-15 01:48:34 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.474: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:43.474: INFO: hostexec-node1-5nkl7 started at 2021-05-15 01:46:52 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.474: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:43.474: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.474: INFO: Container kube-multus ready: true, restart count 1 May 15 01:48:43.474: INFO: prometheus-k8s-0 started at 2021-05-15 00:19:01 +0000 UTC (0+5 container statuses recorded) May 15 01:48:43.474: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 15 01:48:43.474: INFO: Container grafana ready: true, restart count 0 May 15 01:48:43.474: INFO: Container prometheus ready: true, restart count 26 May 15 01:48:43.474: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 15 01:48:43.474: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 15 01:48:43.474: INFO: csi-mockplugin-0 started at 2021-05-15 01:47:22 +0000 UTC (0+3 container statuses recorded) May 15 01:48:43.474: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:48:43.474: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:48:43.474: INFO: Container mock ready: true, restart count 0 May 15 01:48:43.474: INFO: pod-6c919e34-2cba-46c9-86ed-54b69dfbc370 started at 2021-05-15 01:47:59 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.474: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:43.474: INFO: pod-c3357cc4-fc25-4578-af0f-b315e3d2cb0c started at 2021-05-15 01:48:14 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.474: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:43.474: INFO: node-exporter-flvqz started at 2021-05-15 00:18:55 +0000 UTC (0+2 container statuses recorded) May 15 01:48:43.474: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:48:43.474: INFO: Container node-exporter ready: true, restart count 0 May 15 01:48:43.474: INFO: pvc-volume-tester-hwvvq started at 2021-05-15 01:47:34 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.474: INFO: Container volume-tester ready: true, restart count 0 May 15 01:48:43.474: INFO: hostexec-node1-xz74m started at 2021-05-15 01:47:43 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.474: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:43.474: INFO: pod-3f9b2ded-c6a2-495c-8705-d26384e71449 started at 2021-05-15 01:48:43 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.474: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:43.474: INFO: cmk-4s6dm started at 2021-05-15 00:18:54 +0000 UTC (0+2 container statuses recorded) May 15 01:48:43.474: INFO: Container nodereport ready: true, restart count 0 May 15 01:48:43.474: INFO: Container reconcile ready: true, restart count 0 W0515 01:48:43.485794 24 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:48:43.519: INFO: Latency metrics for node node1 May 15 01:48:43.519: INFO: Logging node info for node node2 May 15 01:48:43.521: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 156540 0 2021-05-14 19:58:22 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-3052":"csi-mock-csi-mock-volumes-3052","csi-mock-csi-mock-volumes-8760":"csi-mock-csi-mock-volumes-8760","csi-mock-csi-mock-volumes-9624":"csi-mock-csi-mock-volumes-9624"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:46 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:44:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:46:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:io.kubernetes.storage.mock/node":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:40 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:40 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:40 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:48:40 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:48:43.522: INFO: Logging kubelet events for node node2 May 15 01:48:43.527: INFO: Logging pods the kubelet thinks is on node node2 May 15 01:48:43.537: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:48:43.537: INFO: Init container install-cni ready: true, restart count 1 May 15 01:48:43.537: INFO: Container kube-flannel ready: true, restart count 4 May 15 01:48:43.537: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:48:43.537: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:48:43.537: INFO: Container node-exporter ready: true, restart count 0 May 15 01:48:43.537: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 15 01:48:43.537: INFO: Container tas-controller ready: true, restart count 0 May 15 01:48:43.537: INFO: Container tas-extender ready: true, restart count 0 May 15 01:48:43.537: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 15 01:48:43.538: INFO: Container collectd ready: true, restart count 0 May 15 01:48:43.538: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:48:43.538: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:48:43.538: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.538: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:48:43.538: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.538: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:48:43.538: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 15 01:48:43.538: INFO: Container nodereport ready: true, restart count 0 May 15 01:48:43.538: INFO: Container reconcile ready: true, restart count 0 May 15 01:48:43.538: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.538: INFO: Container cmk-webhook ready: true, restart count 0 May 15 01:48:43.538: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 15 01:48:43.538: INFO: Container discover ready: false, restart count 0 May 15 01:48:43.538: INFO: Container init ready: false, restart count 0 May 15 01:48:43.538: INFO: Container install ready: false, restart count 0 May 15 01:48:43.538: INFO: pod-ephm-test-projected-8bh5 started at 2021-05-15 01:48:29 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.538: INFO: Container test-container-subpath-projected-8bh5 ready: false, restart count 0 May 15 01:48:43.538: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.538: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 15 01:48:43.538: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.538: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:48:43.538: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.538: INFO: Container kube-multus ready: true, restart count 1 May 15 01:48:43.538: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:48:43.538: INFO: Container nginx-proxy ready: true, restart count 2 W0515 01:48:43.550604 24 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:48:43.584: INFO: Latency metrics for node node2 May 15 01:48:43.584: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-5862" for this suite. • Failure in Spec Setup (BeforeEach) [136.853 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: tmpfs] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:205 should be able to mount volume and read from pod1 [BeforeEach] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:228 May 15 01:48:42.991: Unexpected error: <*errors.errorString | 0xc0048a4480>: { s: "pod \"pod-dbfe8699-c076-46f1-bf35-86df11b4addd\" is not Running: timed out waiting for the condition", } pod "pod-dbfe8699-c076-46f1-bf35-86df11b4addd" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 ------------------------------ {"msg":"FAILED [sig-storage] PersistentVolumes-local [Volume type: tmpfs] One pod requesting one prebound PVC should be able to mount volume and read from pod1","total":-1,"completed":2,"skipped":642,"failed":2,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir] One pod requesting one prebound PVC should be able to mount volume and write from pod1","[sig-storage] PersistentVolumes-local [Volume type: tmpfs] One pod requesting one prebound PVC should be able to mount volume and read from pod1"]} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:46:31.029: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: blockfswithformat] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes STEP: Creating block device on node "node1" using path "/tmp/local-volume-test-f69f4092-5f5f-4b14-bdd7-fd3849dab356" May 15 01:46:39.074: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-f69f4092-5f5f-4b14-bdd7-fd3849dab356 && dd if=/dev/zero of=/tmp/local-volume-test-f69f4092-5f5f-4b14-bdd7-fd3849dab356/file bs=4096 count=5120 && losetup -f /tmp/local-volume-test-f69f4092-5f5f-4b14-bdd7-fd3849dab356/file] Namespace:persistent-local-volumes-test-470 PodName:hostexec-node1-frmv7 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:46:39.074: INFO: >>> kubeConfig: /root/.kube/config May 15 01:46:39.216: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c E2E_LOOP_DEV=$(losetup | grep /tmp/local-volume-test-f69f4092-5f5f-4b14-bdd7-fd3849dab356/file | awk '{ print $1 }') 2>&1 > /dev/null && echo ${E2E_LOOP_DEV}] Namespace:persistent-local-volumes-test-470 PodName:hostexec-node1-frmv7 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:46:39.216: INFO: >>> kubeConfig: /root/.kube/config May 15 01:46:39.336: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkfs -t ext4 /dev/loop1 && mount -t ext4 /dev/loop1 /tmp/local-volume-test-f69f4092-5f5f-4b14-bdd7-fd3849dab356 && chmod o+rwx /tmp/local-volume-test-f69f4092-5f5f-4b14-bdd7-fd3849dab356] Namespace:persistent-local-volumes-test-470 PodName:hostexec-node1-frmv7 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:46:39.336: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:46:39.612: INFO: Creating a PV followed by a PVC May 15 01:46:39.619: INFO: Waiting for PV local-pvf28wk to bind to PVC pvc-hnmp7 May 15 01:46:39.619: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-hnmp7] to have phase Bound May 15 01:46:39.621: INFO: PersistentVolumeClaim pvc-hnmp7 found but phase is Pending instead of Bound. May 15 01:46:41.624: INFO: PersistentVolumeClaim pvc-hnmp7 found but phase is Pending instead of Bound. May 15 01:46:43.626: INFO: PersistentVolumeClaim pvc-hnmp7 found and phase=Bound (4.007242565s) May 15 01:46:43.626: INFO: Waiting up to 3m0s for PersistentVolume local-pvf28wk to have phase Bound May 15 01:46:43.629: INFO: PersistentVolume local-pvf28wk found and phase=Bound (2.351134ms) [BeforeEach] One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:211 STEP: Creating pod1 STEP: Creating a pod May 15 01:48:43.650: FAIL: Unexpected error: <*errors.errorString | 0xc00500c230>: { s: "pod \"pod-470e75a2-4056-4cd4-838e-f67be7033ced\" is not Running: timed out waiting for the condition", } pod "pod-470e75a2-4056-4cd4-838e-f67be7033ced" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage.glob..func20.2.3.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 +0x119 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc001d81980) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc001d81980) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc001d81980, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:223 STEP: Deleting pod1 STEP: Deleting pod pod-470e75a2-4056-4cd4-838e-f67be7033ced in namespace persistent-local-volumes-test-470 [AfterEach] [Volume type: blockfswithformat] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:48:43.657: INFO: Deleting PersistentVolumeClaim "pvc-hnmp7" May 15 01:48:43.661: INFO: Deleting PersistentVolume "local-pvf28wk" May 15 01:48:43.665: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c umount /tmp/local-volume-test-f69f4092-5f5f-4b14-bdd7-fd3849dab356] Namespace:persistent-local-volumes-test-470 PodName:hostexec-node1-frmv7 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:48:43.665: INFO: >>> kubeConfig: /root/.kube/config May 15 01:48:44.071: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c E2E_LOOP_DEV=$(losetup | grep /tmp/local-volume-test-f69f4092-5f5f-4b14-bdd7-fd3849dab356/file | awk '{ print $1 }') 2>&1 > /dev/null && echo ${E2E_LOOP_DEV}] Namespace:persistent-local-volumes-test-470 PodName:hostexec-node1-frmv7 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:48:44.071: INFO: >>> kubeConfig: /root/.kube/config STEP: Tear down block device "/dev/loop1" on node "node1" at path /tmp/local-volume-test-f69f4092-5f5f-4b14-bdd7-fd3849dab356/file May 15 01:48:44.240: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c losetup -d /dev/loop1] Namespace:persistent-local-volumes-test-470 PodName:hostexec-node1-frmv7 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:48:44.240: INFO: >>> kubeConfig: /root/.kube/config STEP: Removing the test directory /tmp/local-volume-test-f69f4092-5f5f-4b14-bdd7-fd3849dab356 May 15 01:48:44.351: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-f69f4092-5f5f-4b14-bdd7-fd3849dab356] Namespace:persistent-local-volumes-test-470 PodName:hostexec-node1-frmv7 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:48:44.351: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "persistent-local-volumes-test-470". STEP: Found 13 events. May 15 01:48:44.873: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for hostexec-node1-frmv7: { } Scheduled: Successfully assigned persistent-local-volumes-test-470/hostexec-node1-frmv7 to node1 May 15 01:48:44.873: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-470e75a2-4056-4cd4-838e-f67be7033ced: { } Scheduled: Successfully assigned persistent-local-volumes-test-470/pod-470e75a2-4056-4cd4-838e-f67be7033ced to node1 May 15 01:48:44.873: INFO: At 2021-05-15 01:46:32 +0000 UTC - event for hostexec-node1-frmv7: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 15 01:48:44.873: INFO: At 2021-05-15 01:46:32 +0000 UTC - event for hostexec-node1-frmv7: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 614.142834ms May 15 01:48:44.873: INFO: At 2021-05-15 01:46:32 +0000 UTC - event for hostexec-node1-frmv7: {kubelet node1} Created: Created container agnhost-container May 15 01:48:44.873: INFO: At 2021-05-15 01:46:33 +0000 UTC - event for hostexec-node1-frmv7: {kubelet node1} Started: Started container agnhost-container May 15 01:48:44.873: INFO: At 2021-05-15 01:46:39 +0000 UTC - event for pvc-hnmp7: {persistentvolume-controller } ProvisioningFailed: no volume plugin matched name: kubernetes.io/no-provisioner May 15 01:48:44.873: INFO: At 2021-05-15 01:46:47 +0000 UTC - event for pod-470e75a2-4056-4cd4-838e-f67be7033ced: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:48:44.873: INFO: At 2021-05-15 01:46:47 +0000 UTC - event for pod-470e75a2-4056-4cd4-838e-f67be7033ced: {multus } AddedInterface: Add eth0 [10.244.3.148/24] May 15 01:48:44.873: INFO: At 2021-05-15 01:46:48 +0000 UTC - event for pod-470e75a2-4056-4cd4-838e-f67be7033ced: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:48:44.873: INFO: At 2021-05-15 01:46:48 +0000 UTC - event for pod-470e75a2-4056-4cd4-838e-f67be7033ced: {kubelet node1} Failed: Error: ErrImagePull May 15 01:48:44.873: INFO: At 2021-05-15 01:46:49 +0000 UTC - event for pod-470e75a2-4056-4cd4-838e-f67be7033ced: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 15 01:48:44.873: INFO: At 2021-05-15 01:46:49 +0000 UTC - event for pod-470e75a2-4056-4cd4-838e-f67be7033ced: {kubelet node1} Failed: Error: ImagePullBackOff May 15 01:48:44.875: INFO: POD NODE PHASE GRACE CONDITIONS May 15 01:48:44.875: INFO: hostexec-node1-frmv7 node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:31 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:34 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:34 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:31 +0000 UTC }] May 15 01:48:44.875: INFO: pod-470e75a2-4056-4cd4-838e-f67be7033ced node1 Pending 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:43 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:43 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:43 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:43 +0000 UTC }] May 15 01:48:44.876: INFO: May 15 01:48:44.879: INFO: Logging node info for node master1 May 15 01:48:44.882: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 156512 0 2021-05-14 19:56:35 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:38 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:38 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:38 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:48:38 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[@ :],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:48:44.882: INFO: Logging kubelet events for node master1 May 15 01:48:44.886: INFO: Logging pods the kubelet thinks is on node master1 May 15 01:48:44.894: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:48:44.894: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:48:44.894: INFO: kubernetes-metrics-scraper-678c97765c-fswrn started at 2021-05-15 00:18:49 +0000 UTC (0+1 container statuses recorded) May 15 01:48:44.894: INFO: Container kubernetes-metrics-scraper ready: true, restart count 0 May 15 01:48:44.894: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:48:44.894: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:48:44.894: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:48:44.894: INFO: Container kube-multus ready: true, restart count 1 May 15 01:48:44.894: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 15 01:48:44.894: INFO: Container docker-registry ready: true, restart count 0 May 15 01:48:44.894: INFO: Container nginx ready: true, restart count 0 May 15 01:48:44.894: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 15 01:48:44.894: INFO: Container nfd-controller ready: true, restart count 0 May 15 01:48:44.894: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:48:44.894: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:48:44.894: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:48:44.894: INFO: Init container install-cni ready: true, restart count 0 May 15 01:48:44.894: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:48:44.894: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:48:44.894: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:48:44.894: INFO: Container node-exporter ready: true, restart count 0 May 15 01:48:44.894: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:48:44.894: INFO: Container kube-scheduler ready: true, restart count 0 W0515 01:48:44.905127 21 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:48:44.932: INFO: Latency metrics for node master1 May 15 01:48:44.932: INFO: Logging node info for node master2 May 15 01:48:44.934: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 156493 0 2021-05-14 19:57:04 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:36 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:36 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:36 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:48:36 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:48:44.934: INFO: Logging kubelet events for node master2 May 15 01:48:44.939: INFO: Logging pods the kubelet thinks is on node master2 May 15 01:48:44.948: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:48:44.948: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:48:44.948: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:48:44.948: INFO: Container kube-multus ready: true, restart count 1 May 15 01:48:44.948: INFO: coredns-7677f9bb54-96w24 started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:48:44.948: INFO: Container coredns ready: true, restart count 0 May 15 01:48:44.948: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:48:44.948: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:48:44.948: INFO: Container node-exporter ready: true, restart count 0 May 15 01:48:44.948: INFO: prometheus-operator-5bb8cb9d8f-fqb87 started at 2021-05-15 00:18:49 +0000 UTC (0+2 container statuses recorded) May 15 01:48:44.948: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:48:44.948: INFO: Container prometheus-operator ready: true, restart count 0 May 15 01:48:44.948: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:48:44.948: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:48:44.948: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:48:44.948: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:48:44.948: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:48:44.948: INFO: Container kube-scheduler ready: true, restart count 2 May 15 01:48:44.948: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:48:44.948: INFO: Init container install-cni ready: true, restart count 0 May 15 01:48:44.948: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:48:44.948: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 15 01:48:44.948: INFO: Container autoscaler ready: true, restart count 2 W0515 01:48:44.960947 21 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:48:44.984: INFO: Latency metrics for node master2 May 15 01:48:44.984: INFO: Logging node info for node master3 May 15 01:48:44.987: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 156586 0 2021-05-14 19:57:14 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234767872 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324603904 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:44 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:44 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:44 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:48:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:48:44.987: INFO: Logging kubelet events for node master3 May 15 01:48:44.992: INFO: Logging pods the kubelet thinks is on node master3 May 15 01:48:45.001: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.001: INFO: Container kube-controller-manager ready: true, restart count 3 May 15 01:48:45.001: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.001: INFO: Container kube-scheduler ready: true, restart count 3 May 15 01:48:45.001: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.001: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:48:45.001: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.001: INFO: Container kube-proxy ready: true, restart count 1 May 15 01:48:45.001: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:48:45.001: INFO: Init container install-cni ready: true, restart count 0 May 15 01:48:45.001: INFO: Container kube-flannel ready: true, restart count 2 May 15 01:48:45.001: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.001: INFO: Container kube-multus ready: true, restart count 1 May 15 01:48:45.001: INFO: coredns-7677f9bb54-rpj8c started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.001: INFO: Container coredns ready: true, restart count 0 May 15 01:48:45.001: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:48:45.001: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:48:45.001: INFO: Container node-exporter ready: true, restart count 0 W0515 01:48:45.014180 21 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:48:45.038: INFO: Latency metrics for node master3 May 15 01:48:45.038: INFO: Logging node info for node node1 May 15 01:48:45.041: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 156587 0 2021-05-14 19:58:21 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-7734":"csi-mock-csi-mock-volumes-7734","csi-mock-csi-mock-volumes-9728":"csi-mock-csi-mock-volumes-9728","csi-mock-csi-mock-volumes-9809":"csi-mock-csi-mock-volumes-9809"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:00 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:46:49 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:47:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:44 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:44 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:44 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:48:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[@ :],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:48:45.041: INFO: Logging kubelet events for node node1 May 15 01:48:45.045: INFO: Logging pods the kubelet thinks is on node node1 May 15 01:48:45.062: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.062: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:48:45.062: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4vxcc started at 2021-05-15 00:19:00 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.062: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:48:45.062: INFO: hostexec-node1-frmv7 started at 2021-05-15 01:46:31 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.062: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:45.062: INFO: pod-dbfe8699-c076-46f1-bf35-86df11b4addd started at 2021-05-15 01:46:42 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.062: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:45.062: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.062: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:48:45.062: INFO: pod-a44304e0-d8e2-42b6-8188-544b7d09e42b started at 2021-05-15 01:47:14 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.062: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:45.062: INFO: csi-mockplugin-resizer-0 started at 2021-05-15 01:47:22 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.062: INFO: Container csi-resizer ready: true, restart count 0 May 15 01:48:45.062: INFO: pod-af39b1d5-9c8a-4ff9-938d-ae81bce79a36 started at 2021-05-15 01:48:28 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.062: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:45.062: INFO: hostexec-node1-sfsdm started at 2021-05-15 01:47:47 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.062: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:45.062: INFO: hostexec-node1-sv6vb started at 2021-05-15 01:46:26 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.062: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:45.062: INFO: pod-bec58502-0df0-4903-a0f2-b7940a229fd0 started at 2021-05-15 01:47:58 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.062: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:45.062: INFO: hostexec-node1-b5m77 started at 2021-05-15 01:48:00 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.062: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:45.062: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:48:45.062: INFO: Init container install-cni ready: true, restart count 0 May 15 01:48:45.062: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:48:45.062: INFO: node-feature-discovery-worker-bw8zg started at 2021-05-15 00:18:56 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.062: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:48:45.062: INFO: collectd-mrzps started at 2021-05-15 00:19:22 +0000 UTC (0+3 container statuses recorded) May 15 01:48:45.062: INFO: Container collectd ready: true, restart count 0 May 15 01:48:45.062: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:48:45.062: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:48:45.062: INFO: hostexec-node1-8qtn9 started at 2021-05-15 01:48:34 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.062: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:45.062: INFO: hostexec-node1-5nkl7 started at 2021-05-15 01:46:52 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.062: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:45.062: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.062: INFO: Container kube-multus ready: true, restart count 1 May 15 01:48:45.062: INFO: prometheus-k8s-0 started at 2021-05-15 00:19:01 +0000 UTC (0+5 container statuses recorded) May 15 01:48:45.062: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 15 01:48:45.062: INFO: Container grafana ready: true, restart count 0 May 15 01:48:45.062: INFO: Container prometheus ready: true, restart count 26 May 15 01:48:45.062: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 15 01:48:45.062: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 15 01:48:45.062: INFO: csi-mockplugin-0 started at 2021-05-15 01:47:22 +0000 UTC (0+3 container statuses recorded) May 15 01:48:45.062: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:48:45.062: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:48:45.062: INFO: Container mock ready: true, restart count 0 May 15 01:48:45.062: INFO: pod-6c919e34-2cba-46c9-86ed-54b69dfbc370 started at 2021-05-15 01:47:59 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.062: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:45.062: INFO: pod-c3357cc4-fc25-4578-af0f-b315e3d2cb0c started at 2021-05-15 01:48:14 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.062: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:45.062: INFO: node-exporter-flvqz started at 2021-05-15 00:18:55 +0000 UTC (0+2 container statuses recorded) May 15 01:48:45.062: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:48:45.062: INFO: Container node-exporter ready: true, restart count 0 May 15 01:48:45.062: INFO: pvc-volume-tester-hwvvq started at 2021-05-15 01:47:34 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.062: INFO: Container volume-tester ready: true, restart count 0 May 15 01:48:45.062: INFO: hostexec-node1-xz74m started at 2021-05-15 01:47:43 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.062: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:45.062: INFO: pod-3f9b2ded-c6a2-495c-8705-d26384e71449 started at 2021-05-15 01:48:43 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.062: INFO: Container write-pod ready: false, restart count 0 May 15 01:48:45.062: INFO: cmk-4s6dm started at 2021-05-15 00:18:54 +0000 UTC (0+2 container statuses recorded) May 15 01:48:45.062: INFO: Container nodereport ready: true, restart count 0 May 15 01:48:45.062: INFO: Container reconcile ready: true, restart count 0 May 15 01:48:45.062: INFO: hostexec-node1-wmdkc started at 2021-05-15 01:48:21 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.062: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:48:45.062: INFO: pod-470e75a2-4056-4cd4-838e-f67be7033ced started at 2021-05-15 01:46:43 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.062: INFO: Container write-pod ready: false, restart count 0 W0515 01:48:45.073400 21 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:48:45.112: INFO: Latency metrics for node node1 May 15 01:48:45.112: INFO: Logging node info for node node2 May 15 01:48:45.115: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 156540 0 2021-05-14 19:58:22 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-3052":"csi-mock-csi-mock-volumes-3052","csi-mock-csi-mock-volumes-8760":"csi-mock-csi-mock-volumes-8760","csi-mock-csi-mock-volumes-9624":"csi-mock-csi-mock-volumes-9624"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:46 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:44:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:46:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:io.kubernetes.storage.mock/node":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:40 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:40 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:48:40 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:48:40 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:48:45.115: INFO: Logging kubelet events for node node2 May 15 01:48:45.119: INFO: Logging pods the kubelet thinks is on node node2 May 15 01:48:45.130: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.130: INFO: Container cmk-webhook ready: true, restart count 0 May 15 01:48:45.130: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 15 01:48:45.130: INFO: Container discover ready: false, restart count 0 May 15 01:48:45.130: INFO: Container init ready: false, restart count 0 May 15 01:48:45.130: INFO: Container install ready: false, restart count 0 May 15 01:48:45.130: INFO: pod-ephm-test-projected-8bh5 started at 2021-05-15 01:48:29 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.130: INFO: Container test-container-subpath-projected-8bh5 ready: false, restart count 0 May 15 01:48:45.130: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.130: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:48:45.130: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.130: INFO: Container kube-multus ready: true, restart count 1 May 15 01:48:45.130: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.130: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 15 01:48:45.130: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.130: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:48:45.130: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:48:45.130: INFO: Init container install-cni ready: true, restart count 1 May 15 01:48:45.130: INFO: Container kube-flannel ready: true, restart count 4 May 15 01:48:45.130: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:48:45.130: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:48:45.131: INFO: Container node-exporter ready: true, restart count 0 May 15 01:48:45.131: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 15 01:48:45.131: INFO: Container tas-controller ready: true, restart count 0 May 15 01:48:45.131: INFO: Container tas-extender ready: true, restart count 0 May 15 01:48:45.131: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 15 01:48:45.131: INFO: Container collectd ready: true, restart count 0 May 15 01:48:45.131: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:48:45.131: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:48:45.131: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.131: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:48:45.131: INFO: pod-configmaps-fc419a1f-3248-4355-affc-63afe61e3504 started at 2021-05-15 01:48:43 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.131: INFO: Container configmap-volume-test ready: false, restart count 0 May 15 01:48:45.131: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 15 01:48:45.131: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:48:45.131: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 15 01:48:45.131: INFO: Container nodereport ready: true, restart count 0 May 15 01:48:45.131: INFO: Container reconcile ready: true, restart count 0 W0515 01:48:45.143480 21 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:48:45.186: INFO: Latency metrics for node node2 May 15 01:48:45.186: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-470" for this suite. • Failure in Spec Setup (BeforeEach) [134.164 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: blockfswithformat] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:205 should be able to mount volume and read from pod1 [BeforeEach] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:228 May 15 01:48:43.650: Unexpected error: <*errors.errorString | 0xc00500c230>: { s: "pod \"pod-470e75a2-4056-4cd4-838e-f67be7033ced\" is not Running: timed out waiting for the condition", } pod "pod-470e75a2-4056-4cd4-838e-f67be7033ced" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 ------------------------------ {"msg":"FAILED [sig-storage] PersistentVolumes-local [Volume type: blockfswithformat] One pod requesting one prebound PVC should be able to mount volume and read from pod1","total":-1,"completed":1,"skipped":76,"failed":2,"failures":["[sig-storage] PersistentVolumes-local [Volume type: blockfswithoutformat] One pod requesting one prebound PVC should be able to mount volume and read from pod1","[sig-storage] PersistentVolumes-local [Volume type: blockfswithformat] One pod requesting one prebound PVC should be able to mount volume and read from pod1"]} SSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] GCP Volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:48:45.228: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename gcp-volume STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] GCP Volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/volumes.go:69 May 15 01:48:45.247: INFO: Only supported for node OS distro [gci ubuntu custom] (not debian) [AfterEach] [sig-storage] GCP Volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:48:45.249: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "gcp-volume-1335" for this suite. S [SKIPPING] in Spec Setup (BeforeEach) [0.028 seconds] [sig-storage] GCP Volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/volumes.go:61 NFSv3 [BeforeEach] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/volumes.go:103 should be mountable for NFSv3 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/volumes.go:104 Only supported for node OS distro [gci ubuntu custom] (not debian) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/volumes.go:70 ------------------------------ SSSSSSSS ------------------------------ [BeforeEach] [sig-storage] ConfigMap /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:48:43.711: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename configmap STEP: Waiting for a default service account to be provisioned in namespace [It] should be consumable from pods in volume as non-root with defaultMode and fsGroup set [LinuxOnly] [NodeFeature:FSGroup] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/configmap_volume.go:59 STEP: Creating configMap with name configmap-test-volume-9562e44b-0a2e-4112-a534-6b8f6c039c80 STEP: Creating a pod to test consume configMaps May 15 01:48:43.746: INFO: Waiting up to 5m0s for pod "pod-configmaps-fc419a1f-3248-4355-affc-63afe61e3504" in namespace "configmap-6885" to be "Succeeded or Failed" May 15 01:48:43.751: INFO: Pod "pod-configmaps-fc419a1f-3248-4355-affc-63afe61e3504": Phase="Pending", Reason="", readiness=false. Elapsed: 4.249335ms May 15 01:48:45.755: INFO: Pod "pod-configmaps-fc419a1f-3248-4355-affc-63afe61e3504": Phase="Pending", Reason="", readiness=false. Elapsed: 2.008429959s May 15 01:48:47.759: INFO: Pod "pod-configmaps-fc419a1f-3248-4355-affc-63afe61e3504": Phase="Succeeded", Reason="", readiness=false. Elapsed: 4.012035171s STEP: Saw pod success May 15 01:48:47.759: INFO: Pod "pod-configmaps-fc419a1f-3248-4355-affc-63afe61e3504" satisfied condition "Succeeded or Failed" May 15 01:48:47.762: INFO: Trying to get logs from node node2 pod pod-configmaps-fc419a1f-3248-4355-affc-63afe61e3504 container configmap-volume-test: STEP: delete the pod May 15 01:48:47.777: INFO: Waiting for pod pod-configmaps-fc419a1f-3248-4355-affc-63afe61e3504 to disappear May 15 01:48:47.779: INFO: Pod pod-configmaps-fc419a1f-3248-4355-affc-63afe61e3504 no longer exists [AfterEach] [sig-storage] ConfigMap /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:48:47.779: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "configmap-6885" for this suite. • ------------------------------ {"msg":"PASSED [sig-storage] ConfigMap should be consumable from pods in volume as non-root with defaultMode and fsGroup set [LinuxOnly] [NodeFeature:FSGroup]","total":-1,"completed":3,"skipped":695,"failed":2,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir] One pod requesting one prebound PVC should be able to mount volume and write from pod1","[sig-storage] PersistentVolumes-local [Volume type: tmpfs] One pod requesting one prebound PVC should be able to mount volume and read from pod1"]} SS ------------------------------ [BeforeEach] [sig-storage] Downward API volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:48:47.796: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename downward-api STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] Downward API volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/downwardapi_volume.go:42 [It] should provide podname as non-root with fsgroup [LinuxOnly] [NodeFeature:FSGroup] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/downwardapi_volume.go:92 STEP: Creating a pod to test downward API volume plugin May 15 01:48:47.829: INFO: Waiting up to 5m0s for pod "metadata-volume-8d75979c-152b-4103-8ba0-8550170e1615" in namespace "downward-api-8209" to be "Succeeded or Failed" May 15 01:48:47.832: INFO: Pod "metadata-volume-8d75979c-152b-4103-8ba0-8550170e1615": Phase="Pending", Reason="", readiness=false. Elapsed: 2.803296ms May 15 01:48:49.835: INFO: Pod "metadata-volume-8d75979c-152b-4103-8ba0-8550170e1615": Phase="Pending", Reason="", readiness=false. Elapsed: 2.005940508s May 15 01:48:51.839: INFO: Pod "metadata-volume-8d75979c-152b-4103-8ba0-8550170e1615": Phase="Succeeded", Reason="", readiness=false. Elapsed: 4.009627529s STEP: Saw pod success May 15 01:48:51.839: INFO: Pod "metadata-volume-8d75979c-152b-4103-8ba0-8550170e1615" satisfied condition "Succeeded or Failed" May 15 01:48:51.842: INFO: Trying to get logs from node node2 pod metadata-volume-8d75979c-152b-4103-8ba0-8550170e1615 container client-container: STEP: delete the pod May 15 01:48:51.861: INFO: Waiting for pod metadata-volume-8d75979c-152b-4103-8ba0-8550170e1615 to disappear May 15 01:48:51.863: INFO: Pod metadata-volume-8d75979c-152b-4103-8ba0-8550170e1615 no longer exists [AfterEach] [sig-storage] Downward API volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:48:51.863: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "downward-api-8209" for this suite. • ------------------------------ {"msg":"PASSED [sig-storage] Downward API volume should provide podname as non-root with fsgroup [LinuxOnly] [NodeFeature:FSGroup]","total":-1,"completed":4,"skipped":697,"failed":2,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir] One pod requesting one prebound PVC should be able to mount volume and write from pod1","[sig-storage] PersistentVolumes-local [Volume type: tmpfs] One pod requesting one prebound PVC should be able to mount volume and read from pod1"]} SSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] ConfigMap /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:48:51.925: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename configmap STEP: Waiting for a default service account to be provisioned in namespace [It] should be consumable from pods in volume as non-root with FSGroup [LinuxOnly] [NodeFeature:FSGroup] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/configmap_volume.go:75 STEP: Creating configMap with name configmap-test-volume-53a5b5b4-35b3-4b69-bfa5-c0d30982622e STEP: Creating a pod to test consume configMaps May 15 01:48:51.963: INFO: Waiting up to 5m0s for pod "pod-configmaps-482db520-f315-41d0-96dc-94a126269b83" in namespace "configmap-5838" to be "Succeeded or Failed" May 15 01:48:51.967: INFO: Pod "pod-configmaps-482db520-f315-41d0-96dc-94a126269b83": Phase="Pending", Reason="", readiness=false. Elapsed: 4.335304ms May 15 01:48:53.972: INFO: Pod "pod-configmaps-482db520-f315-41d0-96dc-94a126269b83": Phase="Pending", Reason="", readiness=false. Elapsed: 2.009150184s May 15 01:48:55.978: INFO: Pod "pod-configmaps-482db520-f315-41d0-96dc-94a126269b83": Phase="Succeeded", Reason="", readiness=false. Elapsed: 4.014928034s STEP: Saw pod success May 15 01:48:55.978: INFO: Pod "pod-configmaps-482db520-f315-41d0-96dc-94a126269b83" satisfied condition "Succeeded or Failed" May 15 01:48:55.980: INFO: Trying to get logs from node node2 pod pod-configmaps-482db520-f315-41d0-96dc-94a126269b83 container configmap-volume-test: STEP: delete the pod May 15 01:48:56.003: INFO: Waiting for pod pod-configmaps-482db520-f315-41d0-96dc-94a126269b83 to disappear May 15 01:48:56.005: INFO: Pod pod-configmaps-482db520-f315-41d0-96dc-94a126269b83 no longer exists [AfterEach] [sig-storage] ConfigMap /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:48:56.005: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "configmap-5838" for this suite. • ------------------------------ {"msg":"PASSED [sig-storage] ConfigMap should be consumable from pods in volume as non-root with FSGroup [LinuxOnly] [NodeFeature:FSGroup]","total":-1,"completed":5,"skipped":722,"failed":2,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir] One pod requesting one prebound PVC should be able to mount volume and write from pod1","[sig-storage] PersistentVolumes-local [Volume type: tmpfs] One pod requesting one prebound PVC should be able to mount volume and read from pod1"]} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] Ephemeralstorage /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:48:29.063: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename pv STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] Ephemeralstorage /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/ephemeral_volume.go:49 [It] should allow deletion of pod with invalid volume : configmap /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/ephemeral_volume.go:55 May 15 01:48:59.102: INFO: Deleting pod "pv-4650"/"pod-ephm-test-projected-8bh5" May 15 01:48:59.103: INFO: Deleting pod "pod-ephm-test-projected-8bh5" in namespace "pv-4650" May 15 01:48:59.107: INFO: Wait up to 5m0s for pod "pod-ephm-test-projected-8bh5" to be fully deleted [AfterEach] [sig-storage] Ephemeralstorage /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:49:11.113: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "pv-4650" for this suite. • [SLOW TEST:42.060 seconds] [sig-storage] Ephemeralstorage /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 When pod refers to non-existent ephemeral storage /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/ephemeral_volume.go:53 should allow deletion of pod with invalid volume : configmap /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/ephemeral_volume.go:55 ------------------------------ {"msg":"PASSED [sig-storage] Ephemeralstorage When pod refers to non-existent ephemeral storage should allow deletion of pod with invalid volume : configmap","total":-1,"completed":2,"skipped":87,"failed":2,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir-link] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir-link] One pod requesting one prebound PVC should be able to mount volume and write from pod1"]} SSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:46:52.476: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: dir] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes May 15 01:46:58.531: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-8e46fffc-a01e-45ab-a734-06866e843e80] Namespace:persistent-local-volumes-test-2972 PodName:hostexec-node1-5nkl7 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:46:58.531: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:46:58.644: INFO: Creating a PV followed by a PVC May 15 01:46:58.652: INFO: Waiting for PV local-pvgc5qt to bind to PVC pvc-95r44 May 15 01:46:58.652: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-95r44] to have phase Bound May 15 01:46:58.655: INFO: PersistentVolumeClaim pvc-95r44 found but phase is Pending instead of Bound. May 15 01:47:00.659: INFO: PersistentVolumeClaim pvc-95r44 found but phase is Pending instead of Bound. May 15 01:47:02.663: INFO: PersistentVolumeClaim pvc-95r44 found but phase is Pending instead of Bound. May 15 01:47:04.669: INFO: PersistentVolumeClaim pvc-95r44 found but phase is Pending instead of Bound. May 15 01:47:06.674: INFO: PersistentVolumeClaim pvc-95r44 found but phase is Pending instead of Bound. May 15 01:47:08.676: INFO: PersistentVolumeClaim pvc-95r44 found but phase is Pending instead of Bound. May 15 01:47:10.681: INFO: PersistentVolumeClaim pvc-95r44 found but phase is Pending instead of Bound. May 15 01:47:12.685: INFO: PersistentVolumeClaim pvc-95r44 found but phase is Pending instead of Bound. May 15 01:47:14.692: INFO: PersistentVolumeClaim pvc-95r44 found and phase=Bound (16.040324258s) May 15 01:47:14.692: INFO: Waiting up to 3m0s for PersistentVolume local-pvgc5qt to have phase Bound May 15 01:47:14.697: INFO: PersistentVolume local-pvgc5qt found and phase=Bound (4.255349ms) [It] should be able to write from pod1 and read from pod2 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:251 STEP: Creating pod1 STEP: Creating a pod May 15 01:49:14.720: FAIL: Unexpected error: <*errors.errorString | 0xc004bbaab0>: { s: "pod \"pod-a44304e0-d8e2-42b6-8188-544b7d09e42b\" is not Running: timed out waiting for the condition", } pod "pod-a44304e0-d8e2-42b6-8188-544b7d09e42b" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage.twoPodsReadWriteSerialTest(0xc000df1e40, 0xc003825f00, 0xc004a0cc60) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:764 +0xaf k8s.io/kubernetes/test/e2e/storage.glob..func20.2.5.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:252 +0x45 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000683980) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc000683980) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc000683980, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [Volume type: dir] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:49:14.722: INFO: Deleting PersistentVolumeClaim "pvc-95r44" May 15 01:49:14.727: INFO: Deleting PersistentVolume "local-pvgc5qt" STEP: Removing the test directory May 15 01:49:14.730: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-8e46fffc-a01e-45ab-a734-06866e843e80] Namespace:persistent-local-volumes-test-2972 PodName:hostexec-node1-5nkl7 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:49:14.730: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "persistent-local-volumes-test-2972". STEP: Found 13 events. May 15 01:49:14.856: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for hostexec-node1-5nkl7: { } Scheduled: Successfully assigned persistent-local-volumes-test-2972/hostexec-node1-5nkl7 to node1 May 15 01:49:14.856: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-a44304e0-d8e2-42b6-8188-544b7d09e42b: { } Scheduled: Successfully assigned persistent-local-volumes-test-2972/pod-a44304e0-d8e2-42b6-8188-544b7d09e42b to node1 May 15 01:49:14.856: INFO: At 2021-05-15 01:46:56 +0000 UTC - event for hostexec-node1-5nkl7: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 15 01:49:14.856: INFO: At 2021-05-15 01:46:56 +0000 UTC - event for hostexec-node1-5nkl7: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 535.050899ms May 15 01:49:14.856: INFO: At 2021-05-15 01:46:57 +0000 UTC - event for hostexec-node1-5nkl7: {kubelet node1} Created: Created container agnhost-container May 15 01:49:14.856: INFO: At 2021-05-15 01:46:57 +0000 UTC - event for hostexec-node1-5nkl7: {kubelet node1} Started: Started container agnhost-container May 15 01:49:14.856: INFO: At 2021-05-15 01:46:58 +0000 UTC - event for pvc-95r44: {persistentvolume-controller } ProvisioningFailed: no volume plugin matched name: kubernetes.io/no-provisioner May 15 01:49:14.856: INFO: At 2021-05-15 01:47:17 +0000 UTC - event for pod-a44304e0-d8e2-42b6-8188-544b7d09e42b: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:49:14.856: INFO: At 2021-05-15 01:47:17 +0000 UTC - event for pod-a44304e0-d8e2-42b6-8188-544b7d09e42b: {multus } AddedInterface: Add eth0 [10.244.3.152/24] May 15 01:49:14.856: INFO: At 2021-05-15 01:47:18 +0000 UTC - event for pod-a44304e0-d8e2-42b6-8188-544b7d09e42b: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:49:14.856: INFO: At 2021-05-15 01:47:18 +0000 UTC - event for pod-a44304e0-d8e2-42b6-8188-544b7d09e42b: {kubelet node1} Failed: Error: ErrImagePull May 15 01:49:14.856: INFO: At 2021-05-15 01:47:18 +0000 UTC - event for pod-a44304e0-d8e2-42b6-8188-544b7d09e42b: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 15 01:49:14.856: INFO: At 2021-05-15 01:47:18 +0000 UTC - event for pod-a44304e0-d8e2-42b6-8188-544b7d09e42b: {kubelet node1} Failed: Error: ImagePullBackOff May 15 01:49:14.858: INFO: POD NODE PHASE GRACE CONDITIONS May 15 01:49:14.858: INFO: hostexec-node1-5nkl7 node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:52 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:58 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:58 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:46:52 +0000 UTC }] May 15 01:49:14.858: INFO: pod-a44304e0-d8e2-42b6-8188-544b7d09e42b node1 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:47:14 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:47:14 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:47:14 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:47:14 +0000 UTC }] May 15 01:49:14.858: INFO: May 15 01:49:14.862: INFO: Logging node info for node master1 May 15 01:49:14.864: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 156985 0 2021-05-14 19:56:35 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:08 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:08 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:08 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:49:08 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[@ :],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:49:14.865: INFO: Logging kubelet events for node master1 May 15 01:49:14.868: INFO: Logging pods the kubelet thinks is on node master1 May 15 01:49:14.877: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:49:14.877: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:49:14.877: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:49:14.877: INFO: Init container install-cni ready: true, restart count 0 May 15 01:49:14.877: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:49:14.877: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:49:14.877: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:49:14.877: INFO: Container node-exporter ready: true, restart count 0 May 15 01:49:14.877: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:49:14.877: INFO: Container kube-scheduler ready: true, restart count 0 May 15 01:49:14.877: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 15 01:49:14.877: INFO: Container docker-registry ready: true, restart count 0 May 15 01:49:14.877: INFO: Container nginx ready: true, restart count 0 May 15 01:49:14.877: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 15 01:49:14.877: INFO: Container nfd-controller ready: true, restart count 0 May 15 01:49:14.877: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:49:14.877: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:49:14.877: INFO: kubernetes-metrics-scraper-678c97765c-fswrn started at 2021-05-15 00:18:49 +0000 UTC (0+1 container statuses recorded) May 15 01:49:14.877: INFO: Container kubernetes-metrics-scraper ready: true, restart count 0 May 15 01:49:14.877: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:49:14.877: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:49:14.877: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:49:14.877: INFO: Container kube-multus ready: true, restart count 1 W0515 01:49:14.890871 35 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:49:14.918: INFO: Latency metrics for node master1 May 15 01:49:14.918: INFO: Logging node info for node master2 May 15 01:49:14.921: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 156969 0 2021-05-14 19:57:04 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:06 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:06 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:06 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:49:06 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:49:14.921: INFO: Logging kubelet events for node master2 May 15 01:49:14.927: INFO: Logging pods the kubelet thinks is on node master2 May 15 01:49:14.936: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:49:14.936: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:49:14.936: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:49:14.936: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:49:14.936: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:49:14.936: INFO: Container kube-scheduler ready: true, restart count 2 May 15 01:49:14.936: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:49:14.936: INFO: Init container install-cni ready: true, restart count 0 May 15 01:49:14.936: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:49:14.936: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 15 01:49:14.936: INFO: Container autoscaler ready: true, restart count 2 May 15 01:49:14.936: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:49:14.936: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:49:14.936: INFO: Container node-exporter ready: true, restart count 0 May 15 01:49:14.936: INFO: prometheus-operator-5bb8cb9d8f-fqb87 started at 2021-05-15 00:18:49 +0000 UTC (0+2 container statuses recorded) May 15 01:49:14.936: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:49:14.936: INFO: Container prometheus-operator ready: true, restart count 0 May 15 01:49:14.936: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:49:14.936: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:49:14.936: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:49:14.936: INFO: Container kube-multus ready: true, restart count 1 May 15 01:49:14.936: INFO: coredns-7677f9bb54-96w24 started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:49:14.936: INFO: Container coredns ready: true, restart count 0 W0515 01:49:14.949291 35 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:49:14.973: INFO: Latency metrics for node master2 May 15 01:49:14.973: INFO: Logging node info for node master3 May 15 01:49:14.976: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 157044 0 2021-05-14 19:57:14 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234767872 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324603904 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:14 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:14 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:14 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:49:14 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:49:14.977: INFO: Logging kubelet events for node master3 May 15 01:49:14.982: INFO: Logging pods the kubelet thinks is on node master3 May 15 01:49:14.990: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 15 01:49:14.990: INFO: Container kube-controller-manager ready: true, restart count 3 May 15 01:49:14.990: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:49:14.990: INFO: Container kube-scheduler ready: true, restart count 3 May 15 01:49:14.990: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:49:14.990: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:49:14.990: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:49:14.990: INFO: Container kube-proxy ready: true, restart count 1 May 15 01:49:14.990: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:49:14.990: INFO: Init container install-cni ready: true, restart count 0 May 15 01:49:14.990: INFO: Container kube-flannel ready: true, restart count 2 May 15 01:49:14.990: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:49:14.990: INFO: Container kube-multus ready: true, restart count 1 May 15 01:49:14.990: INFO: coredns-7677f9bb54-rpj8c started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:49:14.990: INFO: Container coredns ready: true, restart count 0 May 15 01:49:14.990: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:49:14.990: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:49:14.990: INFO: Container node-exporter ready: true, restart count 0 W0515 01:49:15.002467 35 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:49:15.033: INFO: Latency metrics for node master3 May 15 01:49:15.033: INFO: Logging node info for node node1 May 15 01:49:15.036: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 157045 0 2021-05-14 19:58:21 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-7734":"csi-mock-csi-mock-volumes-7734","csi-mock-csi-mock-volumes-9728":"csi-mock-csi-mock-volumes-9728","csi-mock-csi-mock-volumes-9809":"csi-mock-csi-mock-volumes-9809"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:00 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:46:49 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:47:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:14 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:14 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:14 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:49:14 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[@ :],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:49:15.037: INFO: Logging kubelet events for node node1 May 15 01:49:15.041: INFO: Logging pods the kubelet thinks is on node node1 May 15 01:49:15.059: INFO: pod-a44304e0-d8e2-42b6-8188-544b7d09e42b started at 2021-05-15 01:47:14 +0000 UTC (0+1 container statuses recorded) May 15 01:49:15.059: INFO: Container write-pod ready: false, restart count 0 May 15 01:49:15.059: INFO: csi-mockplugin-resizer-0 started at 2021-05-15 01:47:22 +0000 UTC (0+1 container statuses recorded) May 15 01:49:15.059: INFO: Container csi-resizer ready: true, restart count 0 May 15 01:49:15.059: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:49:15.059: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:49:15.059: INFO: pod-bec58502-0df0-4903-a0f2-b7940a229fd0 started at 2021-05-15 01:47:58 +0000 UTC (0+1 container statuses recorded) May 15 01:49:15.059: INFO: Container write-pod ready: false, restart count 0 May 15 01:49:15.059: INFO: hostexec-node1-b5m77 started at 2021-05-15 01:48:00 +0000 UTC (0+1 container statuses recorded) May 15 01:49:15.059: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:49:15.059: INFO: pod-af39b1d5-9c8a-4ff9-938d-ae81bce79a36 started at 2021-05-15 01:48:28 +0000 UTC (0+1 container statuses recorded) May 15 01:49:15.059: INFO: Container write-pod ready: false, restart count 0 May 15 01:49:15.059: INFO: hostexec-node1-sfsdm started at 2021-05-15 01:47:47 +0000 UTC (0+1 container statuses recorded) May 15 01:49:15.059: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:49:15.059: INFO: pod-66506bc5-4a99-4e45-8c39-130b362f1eac started at 2021-05-15 01:49:14 +0000 UTC (0+1 container statuses recorded) May 15 01:49:15.059: INFO: Container write-pod ready: false, restart count 0 May 15 01:49:15.059: INFO: collectd-mrzps started at 2021-05-15 00:19:22 +0000 UTC (0+3 container statuses recorded) May 15 01:49:15.059: INFO: Container collectd ready: true, restart count 0 May 15 01:49:15.059: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:49:15.059: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:49:15.059: INFO: hostexec-node1-8qtn9 started at 2021-05-15 01:48:34 +0000 UTC (0+1 container statuses recorded) May 15 01:49:15.059: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:49:15.059: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:49:15.059: INFO: Init container install-cni ready: true, restart count 0 May 15 01:49:15.059: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:49:15.059: INFO: node-feature-discovery-worker-bw8zg started at 2021-05-15 00:18:56 +0000 UTC (0+1 container statuses recorded) May 15 01:49:15.059: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:49:15.059: INFO: csi-mockplugin-0 started at 2021-05-15 01:47:22 +0000 UTC (0+3 container statuses recorded) May 15 01:49:15.059: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:49:15.059: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:49:15.059: INFO: Container mock ready: true, restart count 0 May 15 01:49:15.059: INFO: hostexec-node1-5nkl7 started at 2021-05-15 01:46:52 +0000 UTC (0+1 container statuses recorded) May 15 01:49:15.059: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:49:15.059: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:49:15.059: INFO: Container kube-multus ready: true, restart count 1 May 15 01:49:15.059: INFO: prometheus-k8s-0 started at 2021-05-15 00:19:01 +0000 UTC (0+5 container statuses recorded) May 15 01:49:15.059: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 15 01:49:15.059: INFO: Container grafana ready: true, restart count 0 May 15 01:49:15.059: INFO: Container prometheus ready: true, restart count 26 May 15 01:49:15.059: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 15 01:49:15.059: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 15 01:49:15.059: INFO: hostexec-node1-xz74m started at 2021-05-15 01:47:43 +0000 UTC (0+1 container statuses recorded) May 15 01:49:15.059: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:49:15.059: INFO: pod-6c919e34-2cba-46c9-86ed-54b69dfbc370 started at 2021-05-15 01:47:59 +0000 UTC (0+1 container statuses recorded) May 15 01:49:15.059: INFO: Container write-pod ready: false, restart count 0 May 15 01:49:15.059: INFO: pod-c3357cc4-fc25-4578-af0f-b315e3d2cb0c started at 2021-05-15 01:48:14 +0000 UTC (0+1 container statuses recorded) May 15 01:49:15.059: INFO: Container write-pod ready: false, restart count 0 May 15 01:49:15.059: INFO: node-exporter-flvqz started at 2021-05-15 00:18:55 +0000 UTC (0+2 container statuses recorded) May 15 01:49:15.059: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:49:15.059: INFO: Container node-exporter ready: true, restart count 0 May 15 01:49:15.059: INFO: hostexec-node1-2mw4q started at 2021-05-15 01:48:56 +0000 UTC (0+1 container statuses recorded) May 15 01:49:15.059: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:49:15.059: INFO: pod-3f9b2ded-c6a2-495c-8705-d26384e71449 started at 2021-05-15 01:48:43 +0000 UTC (0+1 container statuses recorded) May 15 01:49:15.060: INFO: Container write-pod ready: false, restart count 0 May 15 01:49:15.060: INFO: cmk-4s6dm started at 2021-05-15 00:18:54 +0000 UTC (0+2 container statuses recorded) May 15 01:49:15.060: INFO: Container nodereport ready: true, restart count 0 May 15 01:49:15.060: INFO: Container reconcile ready: true, restart count 0 May 15 01:49:15.060: INFO: hostexec-node1-wmdkc started at 2021-05-15 01:48:21 +0000 UTC (0+1 container statuses recorded) May 15 01:49:15.060: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:49:15.060: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:49:15.060: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:49:15.060: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4vxcc started at 2021-05-15 00:19:00 +0000 UTC (0+1 container statuses recorded) May 15 01:49:15.060: INFO: Container kube-sriovdp ready: true, restart count 0 W0515 01:49:15.073602 35 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:49:15.112: INFO: Latency metrics for node node1 May 15 01:49:15.112: INFO: Logging node info for node node2 May 15 01:49:15.115: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 156995 0 2021-05-14 19:58:22 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-3052":"csi-mock-csi-mock-volumes-3052","csi-mock-csi-mock-volumes-5678":"csi-mock-csi-mock-volumes-5678","csi-mock-csi-mock-volumes-8760":"csi-mock-csi-mock-volumes-8760","csi-mock-csi-mock-volumes-9624":"csi-mock-csi-mock-volumes-9624"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:46 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:44:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:46:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:io.kubernetes.storage.mock/node":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:10 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:10 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:10 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:49:10 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:49:15.115: INFO: Logging kubelet events for node node2 May 15 01:49:15.120: INFO: Logging pods the kubelet thinks is on node node2 May 15 01:49:15.132: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:49:15.132: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:49:15.132: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:49:15.132: INFO: Init container install-cni ready: true, restart count 1 May 15 01:49:15.132: INFO: Container kube-flannel ready: true, restart count 4 May 15 01:49:15.132: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:49:15.132: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:49:15.132: INFO: Container node-exporter ready: true, restart count 0 May 15 01:49:15.132: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 15 01:49:15.132: INFO: Container tas-controller ready: true, restart count 0 May 15 01:49:15.132: INFO: Container tas-extender ready: true, restart count 0 May 15 01:49:15.132: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 15 01:49:15.132: INFO: Container collectd ready: true, restart count 0 May 15 01:49:15.132: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:49:15.132: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:49:15.132: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 15 01:49:15.132: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:49:15.132: INFO: csi-mockplugin-0 started at 2021-05-15 01:48:45 +0000 UTC (0+3 container statuses recorded) May 15 01:49:15.132: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:49:15.132: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:49:15.132: INFO: Container mock ready: true, restart count 0 May 15 01:49:15.132: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 15 01:49:15.132: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:49:15.132: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 15 01:49:15.132: INFO: Container nodereport ready: true, restart count 0 May 15 01:49:15.132: INFO: Container reconcile ready: true, restart count 0 May 15 01:49:15.132: INFO: pod-ephm-test-projected-lbl6 started at 2021-05-15 01:49:11 +0000 UTC (0+1 container statuses recorded) May 15 01:49:15.132: INFO: Container test-container-subpath-projected-lbl6 ready: false, restart count 0 May 15 01:49:15.132: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 15 01:49:15.132: INFO: Container cmk-webhook ready: true, restart count 0 May 15 01:49:15.132: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 15 01:49:15.132: INFO: Container discover ready: false, restart count 0 May 15 01:49:15.132: INFO: Container init ready: false, restart count 0 May 15 01:49:15.132: INFO: Container install ready: false, restart count 0 May 15 01:49:15.132: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:49:15.132: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:49:15.132: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:49:15.132: INFO: Container kube-multus ready: true, restart count 1 May 15 01:49:15.132: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 15 01:49:15.132: INFO: Container kubernetes-dashboard ready: true, restart count 2 W0515 01:49:15.144468 35 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:49:15.180: INFO: Latency metrics for node node2 May 15 01:49:15.180: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-2972" for this suite. • Failure [142.713 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: dir] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 Two pods mounting a local volume one after the other /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:250 should be able to write from pod1 and read from pod2 [It] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:251 May 15 01:49:14.720: Unexpected error: <*errors.errorString | 0xc004bbaab0>: { s: "pod \"pod-a44304e0-d8e2-42b6-8188-544b7d09e42b\" is not Running: timed out waiting for the condition", } pod "pod-a44304e0-d8e2-42b6-8188-544b7d09e42b" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:764 ------------------------------ {"msg":"FAILED [sig-storage] PersistentVolumes-local [Volume type: dir] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2","total":-1,"completed":1,"skipped":405,"failed":2,"failures":["[sig-storage] PersistentVolumes-local [Volume type: tmpfs] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2"]} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:48:45.274: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename csi-mock-volumes STEP: Waiting for a default service account to be provisioned in namespace [It] unlimited /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:805 STEP: Building a driver namespace object, basename csi-mock-volumes-5678 STEP: Waiting for a default service account to be provisioned in namespace STEP: deploying csi mock driver May 15 01:48:45.345: INFO: creating *v1.ServiceAccount: csi-mock-volumes-5678-2958/csi-attacher May 15 01:48:45.348: INFO: creating *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-5678 May 15 01:48:45.348: INFO: Define cluster role external-attacher-runner-csi-mock-volumes-5678 May 15 01:48:45.351: INFO: creating *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-5678 May 15 01:48:45.354: INFO: creating *v1.Role: csi-mock-volumes-5678-2958/external-attacher-cfg-csi-mock-volumes-5678 May 15 01:48:45.356: INFO: creating *v1.RoleBinding: csi-mock-volumes-5678-2958/csi-attacher-role-cfg May 15 01:48:45.358: INFO: creating *v1.ServiceAccount: csi-mock-volumes-5678-2958/csi-provisioner May 15 01:48:45.361: INFO: creating *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-5678 May 15 01:48:45.361: INFO: Define cluster role external-provisioner-runner-csi-mock-volumes-5678 May 15 01:48:45.364: INFO: creating *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-5678 May 15 01:48:45.366: INFO: creating *v1.Role: csi-mock-volumes-5678-2958/external-provisioner-cfg-csi-mock-volumes-5678 May 15 01:48:45.369: INFO: creating *v1.RoleBinding: csi-mock-volumes-5678-2958/csi-provisioner-role-cfg May 15 01:48:45.371: INFO: creating *v1.ServiceAccount: csi-mock-volumes-5678-2958/csi-resizer May 15 01:48:45.374: INFO: creating *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-5678 May 15 01:48:45.374: INFO: Define cluster role external-resizer-runner-csi-mock-volumes-5678 May 15 01:48:45.377: INFO: creating *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-5678 May 15 01:48:45.380: INFO: creating *v1.Role: csi-mock-volumes-5678-2958/external-resizer-cfg-csi-mock-volumes-5678 May 15 01:48:45.383: INFO: creating *v1.RoleBinding: csi-mock-volumes-5678-2958/csi-resizer-role-cfg May 15 01:48:45.385: INFO: creating *v1.ServiceAccount: csi-mock-volumes-5678-2958/csi-mock May 15 01:48:45.388: INFO: creating *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-5678 May 15 01:48:45.391: INFO: creating *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-5678 May 15 01:48:45.394: INFO: creating *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-5678 May 15 01:48:45.397: INFO: creating *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-5678 May 15 01:48:45.399: INFO: creating *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-5678 May 15 01:48:45.401: INFO: creating *v1.StorageClass: csi-mock-sc-csi-mock-volumes-5678 May 15 01:48:45.404: INFO: creating *v1.StatefulSet: csi-mock-volumes-5678-2958/csi-mockplugin May 15 01:48:45.408: INFO: creating *v1.CSIDriver: csi-mock-csi-mock-volumes-5678 May 15 01:48:45.411: INFO: waiting up to 4m0s for CSIDriver "csi-mock-csi-mock-volumes-5678" May 15 01:48:45.414: INFO: waiting for CSIDriver csi-mock-csi-mock-volumes-5678 to register on node node2 STEP: Creating pod May 15 01:48:50.429: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil May 15 01:48:50.436: INFO: Waiting up to 5m0s for PersistentVolumeClaims [pvc-mbzjz] to have phase Bound May 15 01:48:50.437: INFO: PersistentVolumeClaim pvc-mbzjz found but phase is Pending instead of Bound. May 15 01:48:52.440: INFO: PersistentVolumeClaim pvc-mbzjz found and phase=Bound (2.004756736s) May 15 01:48:56.568: INFO: Deleting pod "pvc-volume-tester-94xh2" in namespace "csi-mock-volumes-5678" May 15 01:48:56.573: INFO: Wait up to 5m0s for pod "pvc-volume-tester-94xh2" to be fully deleted STEP: Checking PVC events May 15 01:49:12.501: INFO: PVC event ADDED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-mbzjz", GenerateName:"pvc-", Namespace:"csi-mock-volumes-5678", SelfLink:"/api/v1/namespaces/csi-mock-volumes-5678/persistentvolumeclaims/pvc-mbzjz", UID:"f488b03c-02c8-42b7-9ecd-45e84685f49e", ResourceVersion:"156788", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63756640130, loc:(*time.Location)(0x770c940)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"e2e.test", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc001913d40), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc001913d60)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"", StorageClassName:(*string)(0xc001721050), VolumeMode:(*v1.PersistentVolumeMode)(0xc001721060), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Pending", AccessModes:[]v1.PersistentVolumeAccessMode(nil), Capacity:v1.ResourceList(nil), Conditions:[]v1.PersistentVolumeClaimCondition(nil)}} May 15 01:49:12.502: INFO: PVC event MODIFIED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-mbzjz", GenerateName:"pvc-", Namespace:"csi-mock-volumes-5678", SelfLink:"/api/v1/namespaces/csi-mock-volumes-5678/persistentvolumeclaims/pvc-mbzjz", UID:"f488b03c-02c8-42b7-9ecd-45e84685f49e", ResourceVersion:"156789", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63756640130, loc:(*time.Location)(0x770c940)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string{"volume.beta.kubernetes.io/storage-provisioner":"csi-mock-csi-mock-volumes-5678"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"e2e.test", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc00236ca40), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc00236ca60)}, v1.ManagedFieldsEntry{Manager:"kube-controller-manager", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc00236ca80), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc00236caa0)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"", StorageClassName:(*string)(0xc0017be000), VolumeMode:(*v1.PersistentVolumeMode)(0xc0017be010), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Pending", AccessModes:[]v1.PersistentVolumeAccessMode(nil), Capacity:v1.ResourceList(nil), Conditions:[]v1.PersistentVolumeClaimCondition(nil)}} May 15 01:49:12.502: INFO: PVC event MODIFIED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-mbzjz", GenerateName:"pvc-", Namespace:"csi-mock-volumes-5678", SelfLink:"/api/v1/namespaces/csi-mock-volumes-5678/persistentvolumeclaims/pvc-mbzjz", UID:"f488b03c-02c8-42b7-9ecd-45e84685f49e", ResourceVersion:"156795", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63756640130, loc:(*time.Location)(0x770c940)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string{"pv.kubernetes.io/bind-completed":"yes", "pv.kubernetes.io/bound-by-controller":"yes", "volume.beta.kubernetes.io/storage-provisioner":"csi-mock-csi-mock-volumes-5678"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"e2e.test", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc001a286a0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc001a286c0)}, v1.ManagedFieldsEntry{Manager:"kube-controller-manager", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc001a286e0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc001a28700)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"pvc-f488b03c-02c8-42b7-9ecd-45e84685f49e", StorageClassName:(*string)(0xc001721b70), VolumeMode:(*v1.PersistentVolumeMode)(0xc001721b80), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Pending", AccessModes:[]v1.PersistentVolumeAccessMode(nil), Capacity:v1.ResourceList(nil), Conditions:[]v1.PersistentVolumeClaimCondition(nil)}} May 15 01:49:12.502: INFO: PVC event MODIFIED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-mbzjz", GenerateName:"pvc-", Namespace:"csi-mock-volumes-5678", SelfLink:"/api/v1/namespaces/csi-mock-volumes-5678/persistentvolumeclaims/pvc-mbzjz", UID:"f488b03c-02c8-42b7-9ecd-45e84685f49e", ResourceVersion:"156796", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63756640130, loc:(*time.Location)(0x770c940)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string{"pv.kubernetes.io/bind-completed":"yes", "pv.kubernetes.io/bound-by-controller":"yes", "volume.beta.kubernetes.io/storage-provisioner":"csi-mock-csi-mock-volumes-5678"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"e2e.test", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc0019e4ee0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0019e4f00)}, v1.ManagedFieldsEntry{Manager:"kube-controller-manager", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc0019e4f20), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0019e4f40)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"pvc-f488b03c-02c8-42b7-9ecd-45e84685f49e", StorageClassName:(*string)(0xc0016cd970), VolumeMode:(*v1.PersistentVolumeMode)(0xc0016cd980), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Bound", AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Capacity:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}, Conditions:[]v1.PersistentVolumeClaimCondition(nil)}} May 15 01:49:12.502: INFO: PVC event MODIFIED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-mbzjz", GenerateName:"pvc-", Namespace:"csi-mock-volumes-5678", SelfLink:"/api/v1/namespaces/csi-mock-volumes-5678/persistentvolumeclaims/pvc-mbzjz", UID:"f488b03c-02c8-42b7-9ecd-45e84685f49e", ResourceVersion:"156998", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63756640130, loc:(*time.Location)(0x770c940)}}, DeletionTimestamp:(*v1.Time)(0xc0019e4f80), DeletionGracePeriodSeconds:(*int64)(0xc002deb718), Labels:map[string]string(nil), Annotations:map[string]string{"pv.kubernetes.io/bind-completed":"yes", "pv.kubernetes.io/bound-by-controller":"yes", "volume.beta.kubernetes.io/storage-provisioner":"csi-mock-csi-mock-volumes-5678"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"e2e.test", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc0019e4fc0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0019e4fe0)}, v1.ManagedFieldsEntry{Manager:"kube-controller-manager", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc0019e5000), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0019e5020)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"pvc-f488b03c-02c8-42b7-9ecd-45e84685f49e", StorageClassName:(*string)(0xc0016cd9c0), VolumeMode:(*v1.PersistentVolumeMode)(0xc0016cd9d0), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Bound", AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Capacity:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}, Conditions:[]v1.PersistentVolumeClaimCondition(nil)}} May 15 01:49:12.502: INFO: PVC event DELETED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-mbzjz", GenerateName:"pvc-", Namespace:"csi-mock-volumes-5678", SelfLink:"/api/v1/namespaces/csi-mock-volumes-5678/persistentvolumeclaims/pvc-mbzjz", UID:"f488b03c-02c8-42b7-9ecd-45e84685f49e", ResourceVersion:"156999", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63756640130, loc:(*time.Location)(0x770c940)}}, DeletionTimestamp:(*v1.Time)(0xc00193c720), DeletionGracePeriodSeconds:(*int64)(0xc000c4b3a8), Labels:map[string]string(nil), Annotations:map[string]string{"pv.kubernetes.io/bind-completed":"yes", "pv.kubernetes.io/bound-by-controller":"yes", "volume.beta.kubernetes.io/storage-provisioner":"csi-mock-csi-mock-volumes-5678"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"e2e.test", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc00193c780), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc00193c7a0)}, v1.ManagedFieldsEntry{Manager:"kube-controller-manager", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc00193c7c0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc00193c7e0)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"pvc-f488b03c-02c8-42b7-9ecd-45e84685f49e", StorageClassName:(*string)(0xc00359ff30), VolumeMode:(*v1.PersistentVolumeMode)(0xc00359ff40), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Bound", AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Capacity:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}, Conditions:[]v1.PersistentVolumeClaimCondition(nil)}} STEP: Deleting pod pvc-volume-tester-94xh2 May 15 01:49:12.502: INFO: Deleting pod "pvc-volume-tester-94xh2" in namespace "csi-mock-volumes-5678" STEP: Deleting claim pvc-mbzjz STEP: Deleting storageclass csi-mock-volumes-5678-sc STEP: Cleaning up resources STEP: deleting the test namespace: csi-mock-volumes-5678 STEP: Waiting for namespaces [csi-mock-volumes-5678] to vanish STEP: uninstalling csi mock driver May 15 01:49:18.524: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-5678-2958/csi-attacher May 15 01:49:18.528: INFO: deleting *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-5678 May 15 01:49:18.533: INFO: deleting *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-5678 May 15 01:49:18.538: INFO: deleting *v1.Role: csi-mock-volumes-5678-2958/external-attacher-cfg-csi-mock-volumes-5678 May 15 01:49:18.542: INFO: deleting *v1.RoleBinding: csi-mock-volumes-5678-2958/csi-attacher-role-cfg May 15 01:49:18.546: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-5678-2958/csi-provisioner May 15 01:49:18.550: INFO: deleting *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-5678 May 15 01:49:18.554: INFO: deleting *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-5678 May 15 01:49:18.558: INFO: deleting *v1.Role: csi-mock-volumes-5678-2958/external-provisioner-cfg-csi-mock-volumes-5678 May 15 01:49:18.562: INFO: deleting *v1.RoleBinding: csi-mock-volumes-5678-2958/csi-provisioner-role-cfg May 15 01:49:18.566: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-5678-2958/csi-resizer May 15 01:49:18.571: INFO: deleting *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-5678 May 15 01:49:18.576: INFO: deleting *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-5678 May 15 01:49:18.581: INFO: deleting *v1.Role: csi-mock-volumes-5678-2958/external-resizer-cfg-csi-mock-volumes-5678 May 15 01:49:18.585: INFO: deleting *v1.RoleBinding: csi-mock-volumes-5678-2958/csi-resizer-role-cfg May 15 01:49:18.589: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-5678-2958/csi-mock May 15 01:49:18.595: INFO: deleting *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-5678 May 15 01:49:18.599: INFO: deleting *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-5678 May 15 01:49:18.604: INFO: deleting *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-5678 May 15 01:49:18.609: INFO: deleting *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-5678 May 15 01:49:18.613: INFO: deleting *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-5678 May 15 01:49:18.618: INFO: deleting *v1.StorageClass: csi-mock-sc-csi-mock-volumes-5678 May 15 01:49:18.623: INFO: deleting *v1.StatefulSet: csi-mock-volumes-5678-2958/csi-mockplugin May 15 01:49:18.628: INFO: deleting *v1.CSIDriver: csi-mock-csi-mock-volumes-5678 STEP: deleting the driver namespace: csi-mock-volumes-5678-2958 STEP: Waiting for namespaces [csi-mock-volumes-5678-2958] to vanish [AfterEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:49:24.643: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "csi-mock-volumes-5678" for this suite. May 15 01:49:24.652: INFO: Namespace csi-mock-volumes-5678 was already deleted STEP: Destroying namespace "csi-mock-volumes-5678-2958" for this suite. May 15 01:49:24.654: INFO: Namespace csi-mock-volumes-5678-2958 was already deleted • [SLOW TEST:39.380 seconds] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 storage capacity /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:752 unlimited /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:805 ------------------------------ {"msg":"PASSED [sig-storage] CSI mock volume storage capacity unlimited","total":-1,"completed":2,"skipped":98,"failed":2,"failures":["[sig-storage] PersistentVolumes-local [Volume type: blockfswithoutformat] One pod requesting one prebound PVC should be able to mount volume and read from pod1","[sig-storage] PersistentVolumes-local [Volume type: blockfswithformat] One pod requesting one prebound PVC should be able to mount volume and read from pod1"]} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:47:22.566: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename csi-mock-volumes STEP: Waiting for a default service account to be provisioned in namespace [It] should expand volume without restarting pod if attach=off, nodeExpansion=on /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:552 STEP: Building a driver namespace object, basename csi-mock-volumes-9728 STEP: Waiting for a default service account to be provisioned in namespace STEP: deploying csi mock driver May 15 01:47:22.622: INFO: creating *v1.ServiceAccount: csi-mock-volumes-9728-9046/csi-attacher May 15 01:47:22.625: INFO: creating *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-9728 May 15 01:47:22.625: INFO: Define cluster role external-attacher-runner-csi-mock-volumes-9728 May 15 01:47:22.628: INFO: creating *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-9728 May 15 01:47:22.631: INFO: creating *v1.Role: csi-mock-volumes-9728-9046/external-attacher-cfg-csi-mock-volumes-9728 May 15 01:47:22.633: INFO: creating *v1.RoleBinding: csi-mock-volumes-9728-9046/csi-attacher-role-cfg May 15 01:47:22.636: INFO: creating *v1.ServiceAccount: csi-mock-volumes-9728-9046/csi-provisioner May 15 01:47:22.639: INFO: creating *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-9728 May 15 01:47:22.639: INFO: Define cluster role external-provisioner-runner-csi-mock-volumes-9728 May 15 01:47:22.641: INFO: creating *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-9728 May 15 01:47:22.644: INFO: creating *v1.Role: csi-mock-volumes-9728-9046/external-provisioner-cfg-csi-mock-volumes-9728 May 15 01:47:22.646: INFO: creating *v1.RoleBinding: csi-mock-volumes-9728-9046/csi-provisioner-role-cfg May 15 01:47:22.648: INFO: creating *v1.ServiceAccount: csi-mock-volumes-9728-9046/csi-resizer May 15 01:47:22.650: INFO: creating *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-9728 May 15 01:47:22.650: INFO: Define cluster role external-resizer-runner-csi-mock-volumes-9728 May 15 01:47:22.652: INFO: creating *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-9728 May 15 01:47:22.654: INFO: creating *v1.Role: csi-mock-volumes-9728-9046/external-resizer-cfg-csi-mock-volumes-9728 May 15 01:47:22.657: INFO: creating *v1.RoleBinding: csi-mock-volumes-9728-9046/csi-resizer-role-cfg May 15 01:47:22.659: INFO: creating *v1.ServiceAccount: csi-mock-volumes-9728-9046/csi-mock May 15 01:47:22.662: INFO: creating *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-9728 May 15 01:47:22.666: INFO: creating *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-9728 May 15 01:47:22.669: INFO: creating *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-9728 May 15 01:47:22.672: INFO: creating *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-9728 May 15 01:47:22.674: INFO: creating *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-9728 May 15 01:47:22.677: INFO: creating *v1.StorageClass: csi-mock-sc-csi-mock-volumes-9728 May 15 01:47:22.679: INFO: creating *v1.StatefulSet: csi-mock-volumes-9728-9046/csi-mockplugin May 15 01:47:22.683: INFO: creating *v1.CSIDriver: csi-mock-csi-mock-volumes-9728 May 15 01:47:22.685: INFO: creating *v1.StatefulSet: csi-mock-volumes-9728-9046/csi-mockplugin-resizer May 15 01:47:22.688: INFO: waiting up to 4m0s for CSIDriver "csi-mock-csi-mock-volumes-9728" May 15 01:47:22.690: INFO: waiting for CSIDriver csi-mock-csi-mock-volumes-9728 to register on node node1 STEP: Creating pod May 15 01:47:32.208: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil May 15 01:47:32.214: INFO: Waiting up to 5m0s for PersistentVolumeClaims [pvc-hgdv2] to have phase Bound May 15 01:47:32.216: INFO: PersistentVolumeClaim pvc-hgdv2 found but phase is Pending instead of Bound. May 15 01:47:34.219: INFO: PersistentVolumeClaim pvc-hgdv2 found and phase=Bound (2.005277838s) STEP: Expanding current pvc STEP: Waiting for persistent volume resize to finish STEP: Waiting for PVC resize to finish STEP: Deleting pod pvc-volume-tester-hwvvq May 15 01:49:08.255: INFO: Deleting pod "pvc-volume-tester-hwvvq" in namespace "csi-mock-volumes-9728" May 15 01:49:08.261: INFO: Wait up to 5m0s for pod "pvc-volume-tester-hwvvq" to be fully deleted STEP: Deleting claim pvc-hgdv2 May 15 01:49:12.273: INFO: Waiting up to 2m0s for PersistentVolume pvc-48f3f6fa-b57e-463f-bd59-f56976b811b5 to get deleted May 15 01:49:12.276: INFO: PersistentVolume pvc-48f3f6fa-b57e-463f-bd59-f56976b811b5 found and phase=Bound (2.489784ms) May 15 01:49:14.279: INFO: PersistentVolume pvc-48f3f6fa-b57e-463f-bd59-f56976b811b5 was removed STEP: Deleting storageclass csi-mock-volumes-9728-sc STEP: Cleaning up resources STEP: deleting the test namespace: csi-mock-volumes-9728 STEP: Waiting for namespaces [csi-mock-volumes-9728] to vanish STEP: uninstalling csi mock driver May 15 01:49:20.297: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-9728-9046/csi-attacher May 15 01:49:20.301: INFO: deleting *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-9728 May 15 01:49:20.305: INFO: deleting *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-9728 May 15 01:49:20.309: INFO: deleting *v1.Role: csi-mock-volumes-9728-9046/external-attacher-cfg-csi-mock-volumes-9728 May 15 01:49:20.314: INFO: deleting *v1.RoleBinding: csi-mock-volumes-9728-9046/csi-attacher-role-cfg May 15 01:49:20.317: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-9728-9046/csi-provisioner May 15 01:49:20.322: INFO: deleting *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-9728 May 15 01:49:20.326: INFO: deleting *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-9728 May 15 01:49:20.331: INFO: deleting *v1.Role: csi-mock-volumes-9728-9046/external-provisioner-cfg-csi-mock-volumes-9728 May 15 01:49:20.335: INFO: deleting *v1.RoleBinding: csi-mock-volumes-9728-9046/csi-provisioner-role-cfg May 15 01:49:20.340: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-9728-9046/csi-resizer May 15 01:49:20.345: INFO: deleting *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-9728 May 15 01:49:20.350: INFO: deleting *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-9728 May 15 01:49:20.355: INFO: deleting *v1.Role: csi-mock-volumes-9728-9046/external-resizer-cfg-csi-mock-volumes-9728 May 15 01:49:20.359: INFO: deleting *v1.RoleBinding: csi-mock-volumes-9728-9046/csi-resizer-role-cfg May 15 01:49:20.363: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-9728-9046/csi-mock May 15 01:49:20.367: INFO: deleting *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-9728 May 15 01:49:20.371: INFO: deleting *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-9728 May 15 01:49:20.375: INFO: deleting *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-9728 May 15 01:49:20.379: INFO: deleting *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-9728 May 15 01:49:20.383: INFO: deleting *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-9728 May 15 01:49:20.387: INFO: deleting *v1.StorageClass: csi-mock-sc-csi-mock-volumes-9728 May 15 01:49:20.392: INFO: deleting *v1.StatefulSet: csi-mock-volumes-9728-9046/csi-mockplugin May 15 01:49:20.396: INFO: deleting *v1.CSIDriver: csi-mock-csi-mock-volumes-9728 May 15 01:49:20.400: INFO: deleting *v1.StatefulSet: csi-mock-volumes-9728-9046/csi-mockplugin-resizer STEP: deleting the driver namespace: csi-mock-volumes-9728-9046 STEP: Waiting for namespaces [csi-mock-volumes-9728-9046] to vanish [AfterEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:49:32.413: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "csi-mock-volumes-9728" for this suite. May 15 01:49:32.421: INFO: Namespace csi-mock-volumes-9728 was already deleted STEP: Destroying namespace "csi-mock-volumes-9728-9046" for this suite. May 15 01:49:32.422: INFO: Namespace csi-mock-volumes-9728-9046 was already deleted • [SLOW TEST:129.856 seconds] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 CSI online volume expansion /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:537 should expand volume without restarting pod if attach=off, nodeExpansion=on /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:552 ------------------------------ {"msg":"PASSED [sig-storage] CSI mock volume CSI online volume expansion should expand volume without restarting pod if attach=off, nodeExpansion=on","total":-1,"completed":3,"skipped":253,"failed":1,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] One pod requesting one prebound PVC should be able to mount volume and write from pod1"]} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] Ephemeralstorage /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:49:11.137: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename pv STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] Ephemeralstorage /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/ephemeral_volume.go:49 [It] should allow deletion of pod with invalid volume : projected /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/ephemeral_volume.go:55 May 15 01:49:41.173: INFO: Deleting pod "pv-5068"/"pod-ephm-test-projected-lbl6" May 15 01:49:41.174: INFO: Deleting pod "pod-ephm-test-projected-lbl6" in namespace "pv-5068" May 15 01:49:41.179: INFO: Wait up to 5m0s for pod "pod-ephm-test-projected-lbl6" to be fully deleted [AfterEach] [sig-storage] Ephemeralstorage /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:49:51.185: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "pv-5068" for this suite. • [SLOW TEST:40.059 seconds] [sig-storage] Ephemeralstorage /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 When pod refers to non-existent ephemeral storage /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/ephemeral_volume.go:53 should allow deletion of pod with invalid volume : projected /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/ephemeral_volume.go:55 ------------------------------ {"msg":"PASSED [sig-storage] Ephemeralstorage When pod refers to non-existent ephemeral storage should allow deletion of pod with invalid volume : projected","total":-1,"completed":3,"skipped":92,"failed":2,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir-link] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir-link] One pod requesting one prebound PVC should be able to mount volume and write from pod1"]} SSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:47:43.811: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: dir-link] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes May 15 01:47:47.857: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir /tmp/local-volume-test-d5846431-456f-42d1-b8a0-12adb0771de6-backend && ln -s /tmp/local-volume-test-d5846431-456f-42d1-b8a0-12adb0771de6-backend /tmp/local-volume-test-d5846431-456f-42d1-b8a0-12adb0771de6] Namespace:persistent-local-volumes-test-679 PodName:hostexec-node1-xz74m ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:47:47.857: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:47:47.974: INFO: Creating a PV followed by a PVC May 15 01:47:47.981: INFO: Waiting for PV local-pvjc57c to bind to PVC pvc-7w4bp May 15 01:47:47.981: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-7w4bp] to have phase Bound May 15 01:47:47.984: INFO: PersistentVolumeClaim pvc-7w4bp found but phase is Pending instead of Bound. May 15 01:47:49.987: INFO: PersistentVolumeClaim pvc-7w4bp found but phase is Pending instead of Bound. May 15 01:47:51.991: INFO: PersistentVolumeClaim pvc-7w4bp found but phase is Pending instead of Bound. May 15 01:47:53.994: INFO: PersistentVolumeClaim pvc-7w4bp found but phase is Pending instead of Bound. May 15 01:47:55.998: INFO: PersistentVolumeClaim pvc-7w4bp found but phase is Pending instead of Bound. May 15 01:47:58.002: INFO: PersistentVolumeClaim pvc-7w4bp found and phase=Bound (10.020559624s) May 15 01:47:58.002: INFO: Waiting up to 3m0s for PersistentVolume local-pvjc57c to have phase Bound May 15 01:47:58.004: INFO: PersistentVolume local-pvjc57c found and phase=Bound (2.617619ms) [It] should be able to write from pod1 and read from pod2 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:251 STEP: Creating pod1 STEP: Creating a pod May 15 01:49:58.027: FAIL: Unexpected error: <*errors.errorString | 0xc003a8db60>: { s: "pod \"pod-bec58502-0df0-4903-a0f2-b7940a229fd0\" is not Running: timed out waiting for the condition", } pod "pod-bec58502-0df0-4903-a0f2-b7940a229fd0" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage.twoPodsReadWriteSerialTest(0xc001b72840, 0xc001075c80, 0xc003b1a840) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:764 +0xaf k8s.io/kubernetes/test/e2e/storage.glob..func20.2.5.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:252 +0x45 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc002904000) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc002904000) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc002904000, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [Volume type: dir-link] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:49:58.029: INFO: Deleting PersistentVolumeClaim "pvc-7w4bp" May 15 01:49:58.034: INFO: Deleting PersistentVolume "local-pvjc57c" STEP: Removing the test directory May 15 01:49:58.039: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-d5846431-456f-42d1-b8a0-12adb0771de6 && rm -r /tmp/local-volume-test-d5846431-456f-42d1-b8a0-12adb0771de6-backend] Namespace:persistent-local-volumes-test-679 PodName:hostexec-node1-xz74m ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:49:58.039: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "persistent-local-volumes-test-679". STEP: Found 16 events. May 15 01:49:58.339: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for hostexec-node1-xz74m: { } Scheduled: Successfully assigned persistent-local-volumes-test-679/hostexec-node1-xz74m to node1 May 15 01:49:58.339: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-bec58502-0df0-4903-a0f2-b7940a229fd0: { } Scheduled: Successfully assigned persistent-local-volumes-test-679/pod-bec58502-0df0-4903-a0f2-b7940a229fd0 to node1 May 15 01:49:58.339: INFO: At 2021-05-15 01:47:44 +0000 UTC - event for hostexec-node1-xz74m: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 15 01:49:58.339: INFO: At 2021-05-15 01:47:44 +0000 UTC - event for hostexec-node1-xz74m: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 509.937069ms May 15 01:49:58.339: INFO: At 2021-05-15 01:47:45 +0000 UTC - event for hostexec-node1-xz74m: {kubelet node1} Created: Created container agnhost-container May 15 01:49:58.339: INFO: At 2021-05-15 01:47:45 +0000 UTC - event for hostexec-node1-xz74m: {kubelet node1} Started: Started container agnhost-container May 15 01:49:58.339: INFO: At 2021-05-15 01:47:47 +0000 UTC - event for pvc-7w4bp: {persistentvolume-controller } ProvisioningFailed: no volume plugin matched name: kubernetes.io/no-provisioner May 15 01:49:58.339: INFO: At 2021-05-15 01:47:59 +0000 UTC - event for pod-bec58502-0df0-4903-a0f2-b7940a229fd0: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:49:58.339: INFO: At 2021-05-15 01:47:59 +0000 UTC - event for pod-bec58502-0df0-4903-a0f2-b7940a229fd0: {multus } AddedInterface: Add eth0 [10.244.3.156/24] May 15 01:49:58.339: INFO: At 2021-05-15 01:48:01 +0000 UTC - event for pod-bec58502-0df0-4903-a0f2-b7940a229fd0: {kubelet node1} Failed: Error: ErrImagePull May 15 01:49:58.339: INFO: At 2021-05-15 01:48:01 +0000 UTC - event for pod-bec58502-0df0-4903-a0f2-b7940a229fd0: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:49:58.339: INFO: At 2021-05-15 01:48:02 +0000 UTC - event for pod-bec58502-0df0-4903-a0f2-b7940a229fd0: {kubelet node1} SandboxChanged: Pod sandbox changed, it will be killed and re-created. May 15 01:49:58.339: INFO: At 2021-05-15 01:48:04 +0000 UTC - event for pod-bec58502-0df0-4903-a0f2-b7940a229fd0: {multus } AddedInterface: Add eth0 [10.244.3.158/24] May 15 01:49:58.339: INFO: At 2021-05-15 01:48:04 +0000 UTC - event for pod-bec58502-0df0-4903-a0f2-b7940a229fd0: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 15 01:49:58.339: INFO: At 2021-05-15 01:48:04 +0000 UTC - event for pod-bec58502-0df0-4903-a0f2-b7940a229fd0: {kubelet node1} Failed: Error: ImagePullBackOff May 15 01:49:58.339: INFO: At 2021-05-15 01:48:21 +0000 UTC - event for pod-bec58502-0df0-4903-a0f2-b7940a229fd0: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:49:58.341: INFO: POD NODE PHASE GRACE CONDITIONS May 15 01:49:58.341: INFO: hostexec-node1-xz74m node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:47:43 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:47:46 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:47:46 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:47:43 +0000 UTC }] May 15 01:49:58.341: INFO: pod-bec58502-0df0-4903-a0f2-b7940a229fd0 node1 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:47:58 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:47:58 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:47:58 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:47:58 +0000 UTC }] May 15 01:49:58.341: INFO: May 15 01:49:58.346: INFO: Logging node info for node master1 May 15 01:49:58.348: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 157613 0 2021-05-14 19:56:35 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:48 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:48 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:48 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:49:48 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[@ :],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:49:58.349: INFO: Logging kubelet events for node master1 May 15 01:49:58.352: INFO: Logging pods the kubelet thinks is on node master1 May 15 01:49:58.362: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.362: INFO: Container nfd-controller ready: true, restart count 0 May 15 01:49:58.362: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.362: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:49:58.362: INFO: kubernetes-metrics-scraper-678c97765c-fswrn started at 2021-05-15 00:18:49 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.362: INFO: Container kubernetes-metrics-scraper ready: true, restart count 0 May 15 01:49:58.362: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.362: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:49:58.362: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.362: INFO: Container kube-multus ready: true, restart count 1 May 15 01:49:58.362: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 15 01:49:58.362: INFO: Container docker-registry ready: true, restart count 0 May 15 01:49:58.362: INFO: Container nginx ready: true, restart count 0 May 15 01:49:58.362: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.362: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:49:58.362: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:49:58.362: INFO: Init container install-cni ready: true, restart count 0 May 15 01:49:58.362: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:49:58.362: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:49:58.362: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:49:58.362: INFO: Container node-exporter ready: true, restart count 0 May 15 01:49:58.362: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.362: INFO: Container kube-scheduler ready: true, restart count 0 W0515 01:49:58.375493 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:49:58.401: INFO: Latency metrics for node master1 May 15 01:49:58.401: INFO: Logging node info for node master2 May 15 01:49:58.404: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 157675 0 2021-05-14 19:57:04 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:56 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:56 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:56 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:49:56 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:49:58.404: INFO: Logging kubelet events for node master2 May 15 01:49:58.409: INFO: Logging pods the kubelet thinks is on node master2 May 15 01:49:58.417: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.417: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:49:58.417: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.417: INFO: Container kube-scheduler ready: true, restart count 2 May 15 01:49:58.417: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:49:58.417: INFO: Init container install-cni ready: true, restart count 0 May 15 01:49:58.417: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:49:58.417: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.417: INFO: Container autoscaler ready: true, restart count 2 May 15 01:49:58.417: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:49:58.417: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:49:58.417: INFO: Container node-exporter ready: true, restart count 0 May 15 01:49:58.417: INFO: prometheus-operator-5bb8cb9d8f-fqb87 started at 2021-05-15 00:18:49 +0000 UTC (0+2 container statuses recorded) May 15 01:49:58.417: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:49:58.417: INFO: Container prometheus-operator ready: true, restart count 0 May 15 01:49:58.417: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.417: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:49:58.417: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.417: INFO: Container kube-multus ready: true, restart count 1 May 15 01:49:58.417: INFO: coredns-7677f9bb54-96w24 started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.418: INFO: Container coredns ready: true, restart count 0 May 15 01:49:58.418: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.418: INFO: Container kube-proxy ready: true, restart count 2 W0515 01:49:58.428854 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:49:58.452: INFO: Latency metrics for node master2 May 15 01:49:58.452: INFO: Logging node info for node master3 May 15 01:49:58.454: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 157651 0 2021-05-14 19:57:14 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234767872 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324603904 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:54 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:54 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:54 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:49:54 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:49:58.454: INFO: Logging kubelet events for node master3 May 15 01:49:58.459: INFO: Logging pods the kubelet thinks is on node master3 May 15 01:49:58.467: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.467: INFO: Container kube-scheduler ready: true, restart count 3 May 15 01:49:58.467: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.467: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:49:58.467: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.467: INFO: Container kube-proxy ready: true, restart count 1 May 15 01:49:58.467: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:49:58.467: INFO: Init container install-cni ready: true, restart count 0 May 15 01:49:58.467: INFO: Container kube-flannel ready: true, restart count 2 May 15 01:49:58.467: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.467: INFO: Container kube-multus ready: true, restart count 1 May 15 01:49:58.467: INFO: coredns-7677f9bb54-rpj8c started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.467: INFO: Container coredns ready: true, restart count 0 May 15 01:49:58.467: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:49:58.467: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:49:58.467: INFO: Container node-exporter ready: true, restart count 0 May 15 01:49:58.467: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.467: INFO: Container kube-controller-manager ready: true, restart count 3 W0515 01:49:58.479683 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:49:58.502: INFO: Latency metrics for node master3 May 15 01:49:58.502: INFO: Logging node info for node node1 May 15 01:49:58.504: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 157659 0 2021-05-14 19:58:21 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1602":"csi-mock-csi-mock-volumes-1602","csi-mock-csi-mock-volumes-7734":"csi-mock-csi-mock-volumes-7734","csi-mock-csi-mock-volumes-9728":"csi-mock-csi-mock-volumes-9728","csi-mock-csi-mock-volumes-9809":"csi-mock-csi-mock-volumes-9809"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:00 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:49:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:49:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:54 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:54 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:54 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:49:54 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[@ :],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:49:58.505: INFO: Logging kubelet events for node node1 May 15 01:49:58.509: INFO: Logging pods the kubelet thinks is on node node1 May 15 01:49:58.532: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:49:58.532: INFO: Init container install-cni ready: true, restart count 0 May 15 01:49:58.532: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:49:58.532: INFO: node-feature-discovery-worker-bw8zg started at 2021-05-15 00:18:56 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.532: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:49:58.532: INFO: collectd-mrzps started at 2021-05-15 00:19:22 +0000 UTC (0+3 container statuses recorded) May 15 01:49:58.532: INFO: Container collectd ready: true, restart count 0 May 15 01:49:58.532: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:49:58.532: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:49:58.532: INFO: hostexec-node1-8qtn9 started at 2021-05-15 01:48:34 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.532: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:49:58.532: INFO: pvc-volume-tester-2xsfm started at 2021-05-15 01:49:26 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.532: INFO: Container volume-tester ready: false, restart count 0 May 15 01:49:58.532: INFO: pod-e5bfb405-29b8-435c-ab5f-54edab17f88c started at 2021-05-15 01:49:43 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.532: INFO: Container write-pod ready: false, restart count 0 May 15 01:49:58.532: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.532: INFO: Container kube-multus ready: true, restart count 1 May 15 01:49:58.532: INFO: prometheus-k8s-0 started at 2021-05-15 00:19:01 +0000 UTC (0+5 container statuses recorded) May 15 01:49:58.532: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 15 01:49:58.532: INFO: Container grafana ready: true, restart count 0 May 15 01:49:58.532: INFO: Container prometheus ready: true, restart count 26 May 15 01:49:58.532: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 15 01:49:58.532: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 15 01:49:58.532: INFO: hostexec-node1-z9htb started at 2021-05-15 01:49:24 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.532: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:49:58.532: INFO: node-exporter-flvqz started at 2021-05-15 00:18:55 +0000 UTC (0+2 container statuses recorded) May 15 01:49:58.532: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:49:58.532: INFO: Container node-exporter ready: true, restart count 0 May 15 01:49:58.532: INFO: csi-mockplugin-0 started at 2021-05-15 01:49:15 +0000 UTC (0+3 container statuses recorded) May 15 01:49:58.532: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:49:58.532: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:49:58.532: INFO: Container mock ready: true, restart count 0 May 15 01:49:58.532: INFO: hostexec-node1-xz74m started at 2021-05-15 01:47:43 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.532: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:49:58.532: INFO: pod-6c919e34-2cba-46c9-86ed-54b69dfbc370 started at 2021-05-15 01:47:59 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.532: INFO: Container write-pod ready: false, restart count 0 May 15 01:49:58.532: INFO: pod-c3357cc4-fc25-4578-af0f-b315e3d2cb0c started at 2021-05-15 01:48:14 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.532: INFO: Container write-pod ready: false, restart count 0 May 15 01:49:58.532: INFO: pod-3f9b2ded-c6a2-495c-8705-d26384e71449 started at 2021-05-15 01:48:43 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.532: INFO: Container write-pod ready: false, restart count 0 May 15 01:49:58.532: INFO: cmk-4s6dm started at 2021-05-15 00:18:54 +0000 UTC (0+2 container statuses recorded) May 15 01:49:58.532: INFO: Container nodereport ready: true, restart count 0 May 15 01:49:58.532: INFO: Container reconcile ready: true, restart count 0 May 15 01:49:58.532: INFO: hostexec-node1-2mw4q started at 2021-05-15 01:48:56 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.532: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:49:58.532: INFO: hostexec-node1-wmdkc started at 2021-05-15 01:48:21 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.532: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:49:58.532: INFO: csi-mockplugin-attacher-0 started at 2021-05-15 01:49:15 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.532: INFO: Container csi-attacher ready: true, restart count 0 May 15 01:49:58.532: INFO: hostexec-node1-8lfk7 started at 2021-05-15 01:49:32 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.532: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:49:58.532: INFO: hostexec-node1-g9jvw started at 2021-05-15 01:49:51 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.532: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:49:58.532: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.532: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:49:58.532: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4vxcc started at 2021-05-15 00:19:00 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.532: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:49:58.532: INFO: pod-92fb967d-00bb-4449-b567-f10d76ce462f started at 2021-05-15 01:49:29 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.532: INFO: Container write-pod ready: false, restart count 0 May 15 01:49:58.532: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.532: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:49:58.532: INFO: hostexec-node1-sfsdm started at 2021-05-15 01:47:47 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.532: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:49:58.532: INFO: pod-66506bc5-4a99-4e45-8c39-130b362f1eac started at 2021-05-15 01:49:14 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.532: INFO: Container write-pod ready: false, restart count 0 May 15 01:49:58.532: INFO: pod-bec58502-0df0-4903-a0f2-b7940a229fd0 started at 2021-05-15 01:47:58 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.532: INFO: Container write-pod ready: false, restart count 0 May 15 01:49:58.532: INFO: hostexec-node1-b5m77 started at 2021-05-15 01:48:00 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.532: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:49:58.532: INFO: pod-af39b1d5-9c8a-4ff9-938d-ae81bce79a36 started at 2021-05-15 01:48:28 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.532: INFO: Container write-pod ready: false, restart count 0 W0515 01:49:58.543793 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:49:58.601: INFO: Latency metrics for node node1 May 15 01:49:58.601: INFO: Logging node info for node node2 May 15 01:49:58.603: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 157621 0 2021-05-14 19:58:22 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-3052":"csi-mock-csi-mock-volumes-3052","csi-mock-csi-mock-volumes-5678":"csi-mock-csi-mock-volumes-5678","csi-mock-csi-mock-volumes-8760":"csi-mock-csi-mock-volumes-8760","csi-mock-csi-mock-volumes-9624":"csi-mock-csi-mock-volumes-9624"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:46 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:44:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:46:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:io.kubernetes.storage.mock/node":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:50 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:50 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:50 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:49:50 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:49:58.603: INFO: Logging kubelet events for node node2 May 15 01:49:58.608: INFO: Logging pods the kubelet thinks is on node node2 May 15 01:49:58.618: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.618: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:49:58.619: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.619: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:49:58.619: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 15 01:49:58.619: INFO: Container nodereport ready: true, restart count 0 May 15 01:49:58.619: INFO: Container reconcile ready: true, restart count 0 May 15 01:49:58.619: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.619: INFO: Container cmk-webhook ready: true, restart count 0 May 15 01:49:58.619: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 15 01:49:58.619: INFO: Container discover ready: false, restart count 0 May 15 01:49:58.619: INFO: Container init ready: false, restart count 0 May 15 01:49:58.619: INFO: Container install ready: false, restart count 0 May 15 01:49:58.619: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.619: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:49:58.619: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.619: INFO: Container kube-multus ready: true, restart count 1 May 15 01:49:58.619: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.619: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 15 01:49:58.619: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:49:58.619: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:49:58.619: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:49:58.619: INFO: Init container install-cni ready: true, restart count 1 May 15 01:49:58.619: INFO: Container kube-flannel ready: true, restart count 4 May 15 01:49:58.619: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:49:58.619: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:49:58.619: INFO: Container node-exporter ready: true, restart count 0 May 15 01:49:58.619: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 15 01:49:58.619: INFO: Container tas-controller ready: true, restart count 0 May 15 01:49:58.619: INFO: Container tas-extender ready: true, restart count 0 May 15 01:49:58.619: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 15 01:49:58.619: INFO: Container collectd ready: true, restart count 0 May 15 01:49:58.619: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:49:58.619: INFO: Container rbac-proxy ready: true, restart count 0 W0515 01:49:58.632101 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:49:58.668: INFO: Latency metrics for node node2 May 15 01:49:58.668: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-679" for this suite. • Failure [134.869 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: dir-link] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 Two pods mounting a local volume one after the other /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:250 should be able to write from pod1 and read from pod2 [It] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:251 May 15 01:49:58.027: Unexpected error: <*errors.errorString | 0xc003a8db60>: { s: "pod \"pod-bec58502-0df0-4903-a0f2-b7940a229fd0\" is not Running: timed out waiting for the condition", } pod "pod-bec58502-0df0-4903-a0f2-b7940a229fd0" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:764 ------------------------------ {"msg":"FAILED [sig-storage] PersistentVolumes-local [Volume type: dir-link] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2","total":-1,"completed":2,"skipped":142,"failed":2,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir-link] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2"]} SSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:47:47.208: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: blockfswithformat] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes STEP: Creating block device on node "node1" using path "/tmp/local-volume-test-6ce16cb0-b05a-4a60-9146-8014610a8277" May 15 01:47:49.254: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-6ce16cb0-b05a-4a60-9146-8014610a8277 && dd if=/dev/zero of=/tmp/local-volume-test-6ce16cb0-b05a-4a60-9146-8014610a8277/file bs=4096 count=5120 && losetup -f /tmp/local-volume-test-6ce16cb0-b05a-4a60-9146-8014610a8277/file] Namespace:persistent-local-volumes-test-8742 PodName:hostexec-node1-sfsdm ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:47:49.254: INFO: >>> kubeConfig: /root/.kube/config May 15 01:47:49.394: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c E2E_LOOP_DEV=$(losetup | grep /tmp/local-volume-test-6ce16cb0-b05a-4a60-9146-8014610a8277/file | awk '{ print $1 }') 2>&1 > /dev/null && echo ${E2E_LOOP_DEV}] Namespace:persistent-local-volumes-test-8742 PodName:hostexec-node1-sfsdm ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:47:49.394: INFO: >>> kubeConfig: /root/.kube/config May 15 01:47:49.507: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkfs -t ext4 /dev/loop2 && mount -t ext4 /dev/loop2 /tmp/local-volume-test-6ce16cb0-b05a-4a60-9146-8014610a8277 && chmod o+rwx /tmp/local-volume-test-6ce16cb0-b05a-4a60-9146-8014610a8277] Namespace:persistent-local-volumes-test-8742 PodName:hostexec-node1-sfsdm ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:47:49.507: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:47:49.789: INFO: Creating a PV followed by a PVC May 15 01:47:49.796: INFO: Waiting for PV local-pvbxlht to bind to PVC pvc-qzq7z May 15 01:47:49.796: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-qzq7z] to have phase Bound May 15 01:47:49.799: INFO: PersistentVolumeClaim pvc-qzq7z found but phase is Pending instead of Bound. May 15 01:47:51.802: INFO: PersistentVolumeClaim pvc-qzq7z found but phase is Pending instead of Bound. May 15 01:47:53.806: INFO: PersistentVolumeClaim pvc-qzq7z found but phase is Pending instead of Bound. May 15 01:47:55.814: INFO: PersistentVolumeClaim pvc-qzq7z found but phase is Pending instead of Bound. May 15 01:47:57.816: INFO: PersistentVolumeClaim pvc-qzq7z found but phase is Pending instead of Bound. May 15 01:47:59.819: INFO: PersistentVolumeClaim pvc-qzq7z found and phase=Bound (10.022696957s) May 15 01:47:59.819: INFO: Waiting up to 3m0s for PersistentVolume local-pvbxlht to have phase Bound May 15 01:47:59.821: INFO: PersistentVolume local-pvbxlht found and phase=Bound (2.216668ms) [It] should be able to write from pod1 and read from pod2 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:251 STEP: Creating pod1 STEP: Creating a pod May 15 01:49:59.847: FAIL: Unexpected error: <*errors.errorString | 0xc00341b600>: { s: "pod \"pod-6c919e34-2cba-46c9-86ed-54b69dfbc370\" is not Running: timed out waiting for the condition", } pod "pod-6c919e34-2cba-46c9-86ed-54b69dfbc370" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage.twoPodsReadWriteSerialTest(0xc001fc49a0, 0xc001d65680, 0xc003448c90) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:764 +0xaf k8s.io/kubernetes/test/e2e/storage.glob..func20.2.5.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:252 +0x45 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc002e7a600) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc002e7a600) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc002e7a600, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [Volume type: blockfswithformat] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:49:59.848: INFO: Deleting PersistentVolumeClaim "pvc-qzq7z" May 15 01:49:59.854: INFO: Deleting PersistentVolume "local-pvbxlht" May 15 01:49:59.858: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c umount /tmp/local-volume-test-6ce16cb0-b05a-4a60-9146-8014610a8277] Namespace:persistent-local-volumes-test-8742 PodName:hostexec-node1-sfsdm ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:49:59.858: INFO: >>> kubeConfig: /root/.kube/config May 15 01:50:00.013: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c E2E_LOOP_DEV=$(losetup | grep /tmp/local-volume-test-6ce16cb0-b05a-4a60-9146-8014610a8277/file | awk '{ print $1 }') 2>&1 > /dev/null && echo ${E2E_LOOP_DEV}] Namespace:persistent-local-volumes-test-8742 PodName:hostexec-node1-sfsdm ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:50:00.013: INFO: >>> kubeConfig: /root/.kube/config STEP: Tear down block device "/dev/loop2" on node "node1" at path /tmp/local-volume-test-6ce16cb0-b05a-4a60-9146-8014610a8277/file May 15 01:50:00.225: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c losetup -d /dev/loop2] Namespace:persistent-local-volumes-test-8742 PodName:hostexec-node1-sfsdm ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:50:00.225: INFO: >>> kubeConfig: /root/.kube/config STEP: Removing the test directory /tmp/local-volume-test-6ce16cb0-b05a-4a60-9146-8014610a8277 May 15 01:50:00.362: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-6ce16cb0-b05a-4a60-9146-8014610a8277] Namespace:persistent-local-volumes-test-8742 PodName:hostexec-node1-sfsdm ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:50:00.363: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "persistent-local-volumes-test-8742". STEP: Found 13 events. May 15 01:50:00.558: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for hostexec-node1-sfsdm: { } Scheduled: Successfully assigned persistent-local-volumes-test-8742/hostexec-node1-sfsdm to node1 May 15 01:50:00.558: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-6c919e34-2cba-46c9-86ed-54b69dfbc370: { } Scheduled: Successfully assigned persistent-local-volumes-test-8742/pod-6c919e34-2cba-46c9-86ed-54b69dfbc370 to node1 May 15 01:50:00.558: INFO: At 2021-05-15 01:47:47 +0000 UTC - event for hostexec-node1-sfsdm: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 15 01:50:00.558: INFO: At 2021-05-15 01:47:48 +0000 UTC - event for hostexec-node1-sfsdm: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 453.542633ms May 15 01:50:00.558: INFO: At 2021-05-15 01:47:48 +0000 UTC - event for hostexec-node1-sfsdm: {kubelet node1} Created: Created container agnhost-container May 15 01:50:00.558: INFO: At 2021-05-15 01:47:48 +0000 UTC - event for hostexec-node1-sfsdm: {kubelet node1} Started: Started container agnhost-container May 15 01:50:00.558: INFO: At 2021-05-15 01:47:49 +0000 UTC - event for pvc-qzq7z: {persistentvolume-controller } ProvisioningFailed: no volume plugin matched name: kubernetes.io/no-provisioner May 15 01:50:00.558: INFO: At 2021-05-15 01:48:01 +0000 UTC - event for pod-6c919e34-2cba-46c9-86ed-54b69dfbc370: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:50:00.558: INFO: At 2021-05-15 01:48:01 +0000 UTC - event for pod-6c919e34-2cba-46c9-86ed-54b69dfbc370: {multus } AddedInterface: Add eth0 [10.244.3.157/24] May 15 01:50:00.558: INFO: At 2021-05-15 01:48:05 +0000 UTC - event for pod-6c919e34-2cba-46c9-86ed-54b69dfbc370: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:50:00.558: INFO: At 2021-05-15 01:48:05 +0000 UTC - event for pod-6c919e34-2cba-46c9-86ed-54b69dfbc370: {kubelet node1} Failed: Error: ErrImagePull May 15 01:50:00.558: INFO: At 2021-05-15 01:48:06 +0000 UTC - event for pod-6c919e34-2cba-46c9-86ed-54b69dfbc370: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 15 01:50:00.558: INFO: At 2021-05-15 01:48:06 +0000 UTC - event for pod-6c919e34-2cba-46c9-86ed-54b69dfbc370: {kubelet node1} Failed: Error: ImagePullBackOff May 15 01:50:00.561: INFO: POD NODE PHASE GRACE CONDITIONS May 15 01:50:00.561: INFO: hostexec-node1-sfsdm node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:47:47 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:47:49 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:47:49 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:47:47 +0000 UTC }] May 15 01:50:00.561: INFO: pod-6c919e34-2cba-46c9-86ed-54b69dfbc370 node1 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:47:59 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:47:59 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:47:59 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:47:59 +0000 UTC }] May 15 01:50:00.561: INFO: May 15 01:50:00.565: INFO: Logging node info for node master1 May 15 01:50:00.568: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 157693 0 2021-05-14 19:56:35 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:58 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:58 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:58 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:49:58 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[@ :],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:50:00.569: INFO: Logging kubelet events for node master1 May 15 01:50:00.573: INFO: Logging pods the kubelet thinks is on node master1 May 15 01:50:00.592: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.592: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:50:00.592: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:50:00.592: INFO: Init container install-cni ready: true, restart count 0 May 15 01:50:00.592: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:50:00.592: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:50:00.592: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:50:00.592: INFO: Container node-exporter ready: true, restart count 0 May 15 01:50:00.592: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.592: INFO: Container kube-scheduler ready: true, restart count 0 May 15 01:50:00.592: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.592: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:50:00.592: INFO: kubernetes-metrics-scraper-678c97765c-fswrn started at 2021-05-15 00:18:49 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.592: INFO: Container kubernetes-metrics-scraper ready: true, restart count 0 May 15 01:50:00.592: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.592: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:50:00.592: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.592: INFO: Container kube-multus ready: true, restart count 1 May 15 01:50:00.592: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 15 01:50:00.592: INFO: Container docker-registry ready: true, restart count 0 May 15 01:50:00.592: INFO: Container nginx ready: true, restart count 0 May 15 01:50:00.592: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.592: INFO: Container nfd-controller ready: true, restart count 0 W0515 01:50:00.605417 29 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:50:00.636: INFO: Latency metrics for node master1 May 15 01:50:00.636: INFO: Logging node info for node master2 May 15 01:50:00.638: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 157675 0 2021-05-14 19:57:04 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:56 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:56 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:56 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:49:56 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:50:00.639: INFO: Logging kubelet events for node master2 May 15 01:50:00.643: INFO: Logging pods the kubelet thinks is on node master2 May 15 01:50:00.652: INFO: prometheus-operator-5bb8cb9d8f-fqb87 started at 2021-05-15 00:18:49 +0000 UTC (0+2 container statuses recorded) May 15 01:50:00.652: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:50:00.652: INFO: Container prometheus-operator ready: true, restart count 0 May 15 01:50:00.652: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.652: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:50:00.652: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.652: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:50:00.652: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.652: INFO: Container kube-scheduler ready: true, restart count 2 May 15 01:50:00.652: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:50:00.652: INFO: Init container install-cni ready: true, restart count 0 May 15 01:50:00.652: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:50:00.652: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.652: INFO: Container autoscaler ready: true, restart count 2 May 15 01:50:00.652: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:50:00.652: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:50:00.652: INFO: Container node-exporter ready: true, restart count 0 May 15 01:50:00.652: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.652: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:50:00.652: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.652: INFO: Container kube-multus ready: true, restart count 1 May 15 01:50:00.652: INFO: coredns-7677f9bb54-96w24 started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.652: INFO: Container coredns ready: true, restart count 0 W0515 01:50:00.662944 29 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:50:00.689: INFO: Latency metrics for node master2 May 15 01:50:00.689: INFO: Logging node info for node master3 May 15 01:50:00.692: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 157651 0 2021-05-14 19:57:14 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234767872 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324603904 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:54 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:54 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:54 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:49:54 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:50:00.692: INFO: Logging kubelet events for node master3 May 15 01:50:00.696: INFO: Logging pods the kubelet thinks is on node master3 May 15 01:50:00.703: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:50:00.703: INFO: Init container install-cni ready: true, restart count 0 May 15 01:50:00.703: INFO: Container kube-flannel ready: true, restart count 2 May 15 01:50:00.703: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.703: INFO: Container kube-multus ready: true, restart count 1 May 15 01:50:00.703: INFO: coredns-7677f9bb54-rpj8c started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.703: INFO: Container coredns ready: true, restart count 0 May 15 01:50:00.703: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:50:00.703: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:50:00.703: INFO: Container node-exporter ready: true, restart count 0 May 15 01:50:00.703: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.703: INFO: Container kube-controller-manager ready: true, restart count 3 May 15 01:50:00.703: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.703: INFO: Container kube-scheduler ready: true, restart count 3 May 15 01:50:00.703: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.703: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:50:00.704: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.704: INFO: Container kube-proxy ready: true, restart count 1 W0515 01:50:00.716832 29 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:50:00.741: INFO: Latency metrics for node master3 May 15 01:50:00.741: INFO: Logging node info for node node1 May 15 01:50:00.744: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 157659 0 2021-05-14 19:58:21 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1602":"csi-mock-csi-mock-volumes-1602","csi-mock-csi-mock-volumes-7734":"csi-mock-csi-mock-volumes-7734","csi-mock-csi-mock-volumes-9728":"csi-mock-csi-mock-volumes-9728","csi-mock-csi-mock-volumes-9809":"csi-mock-csi-mock-volumes-9809"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:00 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:49:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:49:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:54 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:54 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:49:54 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:49:54 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[@ :],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:50:00.744: INFO: Logging kubelet events for node node1 May 15 01:50:00.748: INFO: Logging pods the kubelet thinks is on node node1 May 15 01:50:00.768: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.768: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:50:00.768: INFO: pod-af39b1d5-9c8a-4ff9-938d-ae81bce79a36 started at 2021-05-15 01:48:28 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.768: INFO: Container write-pod ready: false, restart count 0 May 15 01:50:00.768: INFO: hostexec-node1-sfsdm started at 2021-05-15 01:47:47 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.768: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:50:00.768: INFO: pod-66506bc5-4a99-4e45-8c39-130b362f1eac started at 2021-05-15 01:49:14 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.768: INFO: Container write-pod ready: false, restart count 0 May 15 01:50:00.768: INFO: pod-bec58502-0df0-4903-a0f2-b7940a229fd0 started at 2021-05-15 01:47:58 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.768: INFO: Container write-pod ready: false, restart count 0 May 15 01:50:00.768: INFO: hostexec-node1-b5m77 started at 2021-05-15 01:48:00 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.768: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:50:00.768: INFO: pod-e5bfb405-29b8-435c-ab5f-54edab17f88c started at 2021-05-15 01:49:43 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.768: INFO: Container write-pod ready: false, restart count 0 May 15 01:50:00.768: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:50:00.768: INFO: Init container install-cni ready: true, restart count 0 May 15 01:50:00.768: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:50:00.768: INFO: node-feature-discovery-worker-bw8zg started at 2021-05-15 00:18:56 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.768: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:50:00.768: INFO: collectd-mrzps started at 2021-05-15 00:19:22 +0000 UTC (0+3 container statuses recorded) May 15 01:50:00.768: INFO: Container collectd ready: true, restart count 0 May 15 01:50:00.768: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:50:00.768: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:50:00.768: INFO: hostexec-node1-8qtn9 started at 2021-05-15 01:48:34 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.768: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:50:00.768: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.768: INFO: Container kube-multus ready: true, restart count 1 May 15 01:50:00.768: INFO: prometheus-k8s-0 started at 2021-05-15 00:19:01 +0000 UTC (0+5 container statuses recorded) May 15 01:50:00.768: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 15 01:50:00.768: INFO: Container grafana ready: true, restart count 0 May 15 01:50:00.768: INFO: Container prometheus ready: true, restart count 26 May 15 01:50:00.768: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 15 01:50:00.768: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 15 01:50:00.768: INFO: hostexec-node1-z9htb started at 2021-05-15 01:49:24 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.768: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:50:00.768: INFO: pod-6c919e34-2cba-46c9-86ed-54b69dfbc370 started at 2021-05-15 01:47:59 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.768: INFO: Container write-pod ready: false, restart count 0 May 15 01:50:00.768: INFO: pod-c3357cc4-fc25-4578-af0f-b315e3d2cb0c started at 2021-05-15 01:48:14 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.768: INFO: Container write-pod ready: false, restart count 0 May 15 01:50:00.768: INFO: node-exporter-flvqz started at 2021-05-15 00:18:55 +0000 UTC (0+2 container statuses recorded) May 15 01:50:00.768: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:50:00.768: INFO: Container node-exporter ready: true, restart count 0 May 15 01:50:00.768: INFO: csi-mockplugin-0 started at 2021-05-15 01:49:15 +0000 UTC (0+3 container statuses recorded) May 15 01:50:00.768: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:50:00.768: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:50:00.768: INFO: Container mock ready: true, restart count 0 May 15 01:50:00.768: INFO: pod-e94cee9f-0eca-47df-9ce8-1d446f676d35 started at 2021-05-15 01:49:59 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.768: INFO: Container write-pod ready: false, restart count 0 May 15 01:50:00.768: INFO: hostexec-node1-xz74m started at 2021-05-15 01:47:43 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.768: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:50:00.768: INFO: pod-3f9b2ded-c6a2-495c-8705-d26384e71449 started at 2021-05-15 01:48:43 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.768: INFO: Container write-pod ready: false, restart count 0 May 15 01:50:00.768: INFO: cmk-4s6dm started at 2021-05-15 00:18:54 +0000 UTC (0+2 container statuses recorded) May 15 01:50:00.768: INFO: Container nodereport ready: true, restart count 0 May 15 01:50:00.768: INFO: Container reconcile ready: true, restart count 0 May 15 01:50:00.768: INFO: hostexec-node1-2mw4q started at 2021-05-15 01:48:56 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.768: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:50:00.768: INFO: hostexec-node1-wmdkc started at 2021-05-15 01:48:21 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.768: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:50:00.768: INFO: csi-mockplugin-attacher-0 started at 2021-05-15 01:49:15 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.768: INFO: Container csi-attacher ready: true, restart count 0 May 15 01:50:00.768: INFO: hostexec-node1-8lfk7 started at 2021-05-15 01:49:32 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.768: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:50:00.768: INFO: hostexec-node1-g9jvw started at 2021-05-15 01:49:51 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.768: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:50:00.768: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.768: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:50:00.768: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4vxcc started at 2021-05-15 00:19:00 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.768: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:50:00.768: INFO: pod-92fb967d-00bb-4449-b567-f10d76ce462f started at 2021-05-15 01:49:29 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.768: INFO: Container write-pod ready: false, restart count 0 W0515 01:50:00.778139 29 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:50:00.818: INFO: Latency metrics for node node1 May 15 01:50:00.818: INFO: Logging node info for node node2 May 15 01:50:00.822: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 157729 0 2021-05-14 19:58:22 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-3052":"csi-mock-csi-mock-volumes-3052","csi-mock-csi-mock-volumes-5678":"csi-mock-csi-mock-volumes-5678","csi-mock-csi-mock-volumes-8760":"csi-mock-csi-mock-volumes-8760","csi-mock-csi-mock-volumes-9624":"csi-mock-csi-mock-volumes-9624"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:46 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:44:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:46:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:io.kubernetes.storage.mock/node":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:00 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:00 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:00 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:50:00 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:50:00.822: INFO: Logging kubelet events for node node2 May 15 01:50:00.826: INFO: Logging pods the kubelet thinks is on node node2 May 15 01:50:00.840: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.841: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:50:00.841: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 15 01:50:00.841: INFO: Container nodereport ready: true, restart count 0 May 15 01:50:00.841: INFO: Container reconcile ready: true, restart count 0 May 15 01:50:00.841: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.841: INFO: Container cmk-webhook ready: true, restart count 0 May 15 01:50:00.841: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 15 01:50:00.841: INFO: Container discover ready: false, restart count 0 May 15 01:50:00.841: INFO: Container init ready: false, restart count 0 May 15 01:50:00.841: INFO: Container install ready: false, restart count 0 May 15 01:50:00.841: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.841: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:50:00.841: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.841: INFO: Container kube-multus ready: true, restart count 1 May 15 01:50:00.841: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.841: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 15 01:50:00.841: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.841: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:50:00.841: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:50:00.841: INFO: Init container install-cni ready: true, restart count 1 May 15 01:50:00.841: INFO: Container kube-flannel ready: true, restart count 4 May 15 01:50:00.841: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:50:00.841: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:50:00.841: INFO: Container node-exporter ready: true, restart count 0 May 15 01:50:00.841: INFO: pod-host-path-test started at 2021-05-15 01:49:58 +0000 UTC (0+2 container statuses recorded) May 15 01:50:00.841: INFO: Container test-container-1 ready: false, restart count 0 May 15 01:50:00.841: INFO: Container test-container-2 ready: false, restart count 0 May 15 01:50:00.841: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 15 01:50:00.841: INFO: Container tas-controller ready: true, restart count 0 May 15 01:50:00.841: INFO: Container tas-extender ready: true, restart count 0 May 15 01:50:00.841: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 15 01:50:00.841: INFO: Container collectd ready: true, restart count 0 May 15 01:50:00.841: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:50:00.841: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:50:00.841: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 15 01:50:00.841: INFO: Container kube-sriovdp ready: true, restart count 0 W0515 01:50:00.854149 29 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:50:00.889: INFO: Latency metrics for node node2 May 15 01:50:00.889: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-8742" for this suite. • Failure [133.689 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: blockfswithformat] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 Two pods mounting a local volume one after the other /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:250 should be able to write from pod1 and read from pod2 [It] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:251 May 15 01:49:59.847: Unexpected error: <*errors.errorString | 0xc00341b600>: { s: "pod \"pod-6c919e34-2cba-46c9-86ed-54b69dfbc370\" is not Running: timed out waiting for the condition", } pod "pod-6c919e34-2cba-46c9-86ed-54b69dfbc370" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:764 ------------------------------ {"msg":"FAILED [sig-storage] PersistentVolumes-local [Volume type: blockfswithformat] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2","total":-1,"completed":4,"skipped":510,"failed":1,"failures":["[sig-storage] PersistentVolumes-local [Volume type: blockfswithformat] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2"]} SSSSSS ------------------------------ [BeforeEach] [sig-storage] HostPath /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:49:58.714: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename hostpath STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] HostPath /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/host_path.go:37 [It] should support r/w [NodeConformance] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/host_path.go:65 STEP: Creating a pod to test hostPath r/w May 15 01:49:58.751: INFO: Waiting up to 5m0s for pod "pod-host-path-test" in namespace "hostpath-150" to be "Succeeded or Failed" May 15 01:49:58.755: INFO: Pod "pod-host-path-test": Phase="Pending", Reason="", readiness=false. Elapsed: 4.225315ms May 15 01:50:00.758: INFO: Pod "pod-host-path-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2.007831187s May 15 01:50:02.763: INFO: Pod "pod-host-path-test": Phase="Succeeded", Reason="", readiness=false. Elapsed: 4.012103214s STEP: Saw pod success May 15 01:50:02.763: INFO: Pod "pod-host-path-test" satisfied condition "Succeeded or Failed" May 15 01:50:02.766: INFO: Trying to get logs from node node2 pod pod-host-path-test container test-container-2: STEP: delete the pod May 15 01:50:02.782: INFO: Waiting for pod pod-host-path-test to disappear May 15 01:50:02.784: INFO: Pod pod-host-path-test no longer exists [AfterEach] [sig-storage] HostPath /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:50:02.784: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "hostpath-150" for this suite. • ------------------------------ {"msg":"PASSED [sig-storage] HostPath should support r/w [NodeConformance]","total":-1,"completed":3,"skipped":155,"failed":2,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir-link] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2"]} SSSS ------------------------------ [BeforeEach] [sig-storage] Projected secret /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:50:00.918: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename projected STEP: Waiting for a default service account to be provisioned in namespace [It] should be able to mount in a volume regardless of a different secret existing with same name in different namespace [NodeConformance] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/projected_secret.go:90 STEP: Creating projection with secret that has name projected-secret-test-1def645b-1478-40ba-b53f-e1d76bbc6aac STEP: Creating a pod to test consume secrets May 15 01:50:00.980: INFO: Waiting up to 5m0s for pod "pod-projected-secrets-c8e6fc02-ae3f-4e95-a2b1-c2b6ecc71616" in namespace "projected-9314" to be "Succeeded or Failed" May 15 01:50:00.982: INFO: Pod "pod-projected-secrets-c8e6fc02-ae3f-4e95-a2b1-c2b6ecc71616": Phase="Pending", Reason="", readiness=false. Elapsed: 1.623147ms May 15 01:50:02.985: INFO: Pod "pod-projected-secrets-c8e6fc02-ae3f-4e95-a2b1-c2b6ecc71616": Phase="Pending", Reason="", readiness=false. Elapsed: 2.004757813s May 15 01:50:04.990: INFO: Pod "pod-projected-secrets-c8e6fc02-ae3f-4e95-a2b1-c2b6ecc71616": Phase="Succeeded", Reason="", readiness=false. Elapsed: 4.009901186s STEP: Saw pod success May 15 01:50:04.990: INFO: Pod "pod-projected-secrets-c8e6fc02-ae3f-4e95-a2b1-c2b6ecc71616" satisfied condition "Succeeded or Failed" May 15 01:50:04.993: INFO: Trying to get logs from node node2 pod pod-projected-secrets-c8e6fc02-ae3f-4e95-a2b1-c2b6ecc71616 container projected-secret-volume-test: STEP: delete the pod May 15 01:50:05.007: INFO: Waiting for pod pod-projected-secrets-c8e6fc02-ae3f-4e95-a2b1-c2b6ecc71616 to disappear May 15 01:50:05.009: INFO: Pod pod-projected-secrets-c8e6fc02-ae3f-4e95-a2b1-c2b6ecc71616 no longer exists [AfterEach] [sig-storage] Projected secret /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:50:05.009: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "projected-9314" for this suite. STEP: Destroying namespace "secret-namespace-956" for this suite. • ------------------------------ {"msg":"PASSED [sig-storage] Projected secret should be able to mount in a volume regardless of a different secret existing with same name in different namespace [NodeConformance]","total":-1,"completed":5,"skipped":516,"failed":1,"failures":["[sig-storage] PersistentVolumes-local [Volume type: blockfswithformat] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2"]} SSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:48:00.481: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: block] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes STEP: Creating block device on node "node1" using path "/tmp/local-volume-test-e76b4752-3733-4625-926b-59de27fe9802" May 15 01:48:08.528: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-e76b4752-3733-4625-926b-59de27fe9802 && dd if=/dev/zero of=/tmp/local-volume-test-e76b4752-3733-4625-926b-59de27fe9802/file bs=4096 count=5120 && losetup -f /tmp/local-volume-test-e76b4752-3733-4625-926b-59de27fe9802/file] Namespace:persistent-local-volumes-test-4698 PodName:hostexec-node1-b5m77 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:48:08.528: INFO: >>> kubeConfig: /root/.kube/config May 15 01:48:08.672: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c E2E_LOOP_DEV=$(losetup | grep /tmp/local-volume-test-e76b4752-3733-4625-926b-59de27fe9802/file | awk '{ print $1 }') 2>&1 > /dev/null && echo ${E2E_LOOP_DEV}] Namespace:persistent-local-volumes-test-4698 PodName:hostexec-node1-b5m77 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:48:08.672: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:48:08.799: INFO: Creating a PV followed by a PVC May 15 01:48:08.807: INFO: Waiting for PV local-pv6dwnx to bind to PVC pvc-fzf56 May 15 01:48:08.807: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-fzf56] to have phase Bound May 15 01:48:08.809: INFO: PersistentVolumeClaim pvc-fzf56 found but phase is Pending instead of Bound. May 15 01:48:10.814: INFO: PersistentVolumeClaim pvc-fzf56 found but phase is Pending instead of Bound. May 15 01:48:12.818: INFO: PersistentVolumeClaim pvc-fzf56 found but phase is Pending instead of Bound. May 15 01:48:14.822: INFO: PersistentVolumeClaim pvc-fzf56 found and phase=Bound (6.014851731s) May 15 01:48:14.822: INFO: Waiting up to 3m0s for PersistentVolume local-pv6dwnx to have phase Bound May 15 01:48:14.824: INFO: PersistentVolume local-pv6dwnx found and phase=Bound (2.524633ms) [BeforeEach] One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:211 STEP: Creating pod1 STEP: Creating a pod May 15 01:50:14.853: FAIL: Unexpected error: <*errors.errorString | 0xc003e472f0>: { s: "pod \"pod-c3357cc4-fc25-4578-af0f-b315e3d2cb0c\" is not Running: timed out waiting for the condition", } pod "pod-c3357cc4-fc25-4578-af0f-b315e3d2cb0c" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage.glob..func20.2.3.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 +0x119 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000e53800) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc000e53800) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc000e53800, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:223 STEP: Deleting pod1 STEP: Deleting pod pod-c3357cc4-fc25-4578-af0f-b315e3d2cb0c in namespace persistent-local-volumes-test-4698 [AfterEach] [Volume type: block] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:50:14.859: INFO: Deleting PersistentVolumeClaim "pvc-fzf56" May 15 01:50:14.863: INFO: Deleting PersistentVolume "local-pv6dwnx" May 15 01:50:14.867: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c E2E_LOOP_DEV=$(losetup | grep /tmp/local-volume-test-e76b4752-3733-4625-926b-59de27fe9802/file | awk '{ print $1 }') 2>&1 > /dev/null && echo ${E2E_LOOP_DEV}] Namespace:persistent-local-volumes-test-4698 PodName:hostexec-node1-b5m77 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:50:14.867: INFO: >>> kubeConfig: /root/.kube/config STEP: Tear down block device "/dev/loop3" on node "node1" at path /tmp/local-volume-test-e76b4752-3733-4625-926b-59de27fe9802/file May 15 01:50:14.982: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c losetup -d /dev/loop3] Namespace:persistent-local-volumes-test-4698 PodName:hostexec-node1-b5m77 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:50:14.982: INFO: >>> kubeConfig: /root/.kube/config STEP: Removing the test directory /tmp/local-volume-test-e76b4752-3733-4625-926b-59de27fe9802 May 15 01:50:15.095: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-e76b4752-3733-4625-926b-59de27fe9802] Namespace:persistent-local-volumes-test-4698 PodName:hostexec-node1-b5m77 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:50:15.095: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "persistent-local-volumes-test-4698". STEP: Found 17 events. May 15 01:50:15.230: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for hostexec-node1-b5m77: { } Scheduled: Successfully assigned persistent-local-volumes-test-4698/hostexec-node1-b5m77 to node1 May 15 01:50:15.230: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-c3357cc4-fc25-4578-af0f-b315e3d2cb0c: { } Scheduled: Successfully assigned persistent-local-volumes-test-4698/pod-c3357cc4-fc25-4578-af0f-b315e3d2cb0c to node1 May 15 01:50:15.230: INFO: At 2021-05-15 01:48:02 +0000 UTC - event for hostexec-node1-b5m77: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 15 01:50:15.230: INFO: At 2021-05-15 01:48:06 +0000 UTC - event for hostexec-node1-b5m77: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 4.272205261s May 15 01:50:15.231: INFO: At 2021-05-15 01:48:06 +0000 UTC - event for hostexec-node1-b5m77: {kubelet node1} Created: Created container agnhost-container May 15 01:50:15.231: INFO: At 2021-05-15 01:48:06 +0000 UTC - event for hostexec-node1-b5m77: {kubelet node1} Started: Started container agnhost-container May 15 01:50:15.231: INFO: At 2021-05-15 01:48:08 +0000 UTC - event for pvc-fzf56: {persistentvolume-controller } ProvisioningFailed: no volume plugin matched name: kubernetes.io/no-provisioner May 15 01:50:15.231: INFO: At 2021-05-15 01:48:15 +0000 UTC - event for pod-c3357cc4-fc25-4578-af0f-b315e3d2cb0c: {kubelet node1} SuccessfulMountVolume: MapVolume.MapPodDevice succeeded for volume "local-pv6dwnx" volumeMapPath "/var/lib/kubelet/pods/7dc29d33-f4e0-4f0c-8f15-8d8e1bfa9a50/volumeDevices/kubernetes.io~local-volume" May 15 01:50:15.231: INFO: At 2021-05-15 01:48:15 +0000 UTC - event for pod-c3357cc4-fc25-4578-af0f-b315e3d2cb0c: {kubelet node1} SuccessfulMountVolume: MapVolume.MapPodDevice succeeded for volume "local-pv6dwnx" globalMapPath "/var/lib/kubelet/plugins/kubernetes.io~local-volume/volumeDevices/local-pv6dwnx" May 15 01:50:15.231: INFO: At 2021-05-15 01:48:16 +0000 UTC - event for pod-c3357cc4-fc25-4578-af0f-b315e3d2cb0c: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:50:15.231: INFO: At 2021-05-15 01:48:16 +0000 UTC - event for pod-c3357cc4-fc25-4578-af0f-b315e3d2cb0c: {multus } AddedInterface: Add eth0 [10.244.3.159/24] May 15 01:50:15.231: INFO: At 2021-05-15 01:48:17 +0000 UTC - event for pod-c3357cc4-fc25-4578-af0f-b315e3d2cb0c: {kubelet node1} Failed: Error: ErrImagePull May 15 01:50:15.231: INFO: At 2021-05-15 01:48:17 +0000 UTC - event for pod-c3357cc4-fc25-4578-af0f-b315e3d2cb0c: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:50:15.231: INFO: At 2021-05-15 01:48:18 +0000 UTC - event for pod-c3357cc4-fc25-4578-af0f-b315e3d2cb0c: {kubelet node1} SandboxChanged: Pod sandbox changed, it will be killed and re-created. May 15 01:50:15.231: INFO: At 2021-05-15 01:48:20 +0000 UTC - event for pod-c3357cc4-fc25-4578-af0f-b315e3d2cb0c: {multus } AddedInterface: Add eth0 [10.244.3.160/24] May 15 01:50:15.231: INFO: At 2021-05-15 01:48:20 +0000 UTC - event for pod-c3357cc4-fc25-4578-af0f-b315e3d2cb0c: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 15 01:50:15.231: INFO: At 2021-05-15 01:48:20 +0000 UTC - event for pod-c3357cc4-fc25-4578-af0f-b315e3d2cb0c: {kubelet node1} Failed: Error: ImagePullBackOff May 15 01:50:15.234: INFO: POD NODE PHASE GRACE CONDITIONS May 15 01:50:15.234: INFO: hostexec-node1-b5m77 node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:48:00 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:48:07 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:48:07 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:48:00 +0000 UTC }] May 15 01:50:15.234: INFO: pod-c3357cc4-fc25-4578-af0f-b315e3d2cb0c node1 Pending 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:48:14 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:48:14 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:48:14 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:48:14 +0000 UTC }] May 15 01:50:15.234: INFO: May 15 01:50:15.239: INFO: Logging node info for node master1 May 15 01:50:15.241: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 157997 0 2021-05-14 19:56:35 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:08 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:08 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:08 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:50:08 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[@ :],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:50:15.242: INFO: Logging kubelet events for node master1 May 15 01:50:15.246: INFO: Logging pods the kubelet thinks is on node master1 May 15 01:50:15.255: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.255: INFO: Container kube-multus ready: true, restart count 1 May 15 01:50:15.255: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 15 01:50:15.255: INFO: Container docker-registry ready: true, restart count 0 May 15 01:50:15.255: INFO: Container nginx ready: true, restart count 0 May 15 01:50:15.255: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.255: INFO: Container nfd-controller ready: true, restart count 0 May 15 01:50:15.255: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.255: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:50:15.255: INFO: kubernetes-metrics-scraper-678c97765c-fswrn started at 2021-05-15 00:18:49 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.255: INFO: Container kubernetes-metrics-scraper ready: true, restart count 0 May 15 01:50:15.255: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.255: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:50:15.255: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.255: INFO: Container kube-scheduler ready: true, restart count 0 May 15 01:50:15.255: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.255: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:50:15.255: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:50:15.255: INFO: Init container install-cni ready: true, restart count 0 May 15 01:50:15.255: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:50:15.255: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:50:15.255: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:50:15.255: INFO: Container node-exporter ready: true, restart count 0 W0515 01:50:15.267779 28 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:50:15.293: INFO: Latency metrics for node master1 May 15 01:50:15.293: INFO: Logging node info for node master2 May 15 01:50:15.296: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 157931 0 2021-05-14 19:57:04 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:06 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:06 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:06 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:50:06 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:50:15.297: INFO: Logging kubelet events for node master2 May 15 01:50:15.301: INFO: Logging pods the kubelet thinks is on node master2 May 15 01:50:15.310: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.310: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:50:15.310: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.311: INFO: Container kube-multus ready: true, restart count 1 May 15 01:50:15.311: INFO: coredns-7677f9bb54-96w24 started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.311: INFO: Container coredns ready: true, restart count 0 May 15 01:50:15.311: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.311: INFO: Container autoscaler ready: true, restart count 2 May 15 01:50:15.311: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:50:15.311: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:50:15.311: INFO: Container node-exporter ready: true, restart count 0 May 15 01:50:15.311: INFO: prometheus-operator-5bb8cb9d8f-fqb87 started at 2021-05-15 00:18:49 +0000 UTC (0+2 container statuses recorded) May 15 01:50:15.311: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:50:15.311: INFO: Container prometheus-operator ready: true, restart count 0 May 15 01:50:15.311: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.311: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:50:15.311: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.311: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:50:15.311: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.311: INFO: Container kube-scheduler ready: true, restart count 2 May 15 01:50:15.311: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:50:15.311: INFO: Init container install-cni ready: true, restart count 0 May 15 01:50:15.311: INFO: Container kube-flannel ready: true, restart count 1 W0515 01:50:15.322737 28 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:50:15.348: INFO: Latency metrics for node master2 May 15 01:50:15.348: INFO: Logging node info for node master3 May 15 01:50:15.350: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 158117 0 2021-05-14 19:57:14 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234767872 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324603904 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:14 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:14 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:14 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:50:14 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:50:15.351: INFO: Logging kubelet events for node master3 May 15 01:50:15.355: INFO: Logging pods the kubelet thinks is on node master3 May 15 01:50:15.362: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.362: INFO: Container kube-multus ready: true, restart count 1 May 15 01:50:15.362: INFO: coredns-7677f9bb54-rpj8c started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.362: INFO: Container coredns ready: true, restart count 0 May 15 01:50:15.362: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:50:15.362: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:50:15.362: INFO: Container node-exporter ready: true, restart count 0 May 15 01:50:15.362: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.362: INFO: Container kube-controller-manager ready: true, restart count 3 May 15 01:50:15.362: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.362: INFO: Container kube-scheduler ready: true, restart count 3 May 15 01:50:15.362: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.362: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:50:15.362: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.362: INFO: Container kube-proxy ready: true, restart count 1 May 15 01:50:15.362: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:50:15.362: INFO: Init container install-cni ready: true, restart count 0 May 15 01:50:15.362: INFO: Container kube-flannel ready: true, restart count 2 W0515 01:50:15.375291 28 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:50:15.403: INFO: Latency metrics for node master3 May 15 01:50:15.403: INFO: Logging node info for node node1 May 15 01:50:15.405: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 158129 0 2021-05-14 19:58:21 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1602":"csi-mock-csi-mock-volumes-1602","csi-mock-csi-mock-volumes-7734":"csi-mock-csi-mock-volumes-7734","csi-mock-csi-mock-volumes-9728":"csi-mock-csi-mock-volumes-9728","csi-mock-csi-mock-volumes-9809":"csi-mock-csi-mock-volumes-9809"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:00 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:49:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:49:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:15 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:15 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:15 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:50:15 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[@ :],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:50:15.406: INFO: Logging kubelet events for node node1 May 15 01:50:15.410: INFO: Logging pods the kubelet thinks is on node node1 May 15 01:50:15.427: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.427: INFO: Container kube-multus ready: true, restart count 1 May 15 01:50:15.427: INFO: prometheus-k8s-0 started at 2021-05-15 00:19:01 +0000 UTC (0+5 container statuses recorded) May 15 01:50:15.427: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 15 01:50:15.427: INFO: Container grafana ready: true, restart count 0 May 15 01:50:15.427: INFO: Container prometheus ready: true, restart count 26 May 15 01:50:15.427: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 15 01:50:15.427: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 15 01:50:15.427: INFO: hostexec-node1-z9htb started at 2021-05-15 01:49:24 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.427: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:50:15.427: INFO: node-exporter-flvqz started at 2021-05-15 00:18:55 +0000 UTC (0+2 container statuses recorded) May 15 01:50:15.427: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:50:15.427: INFO: Container node-exporter ready: true, restart count 0 May 15 01:50:15.427: INFO: csi-mockplugin-0 started at 2021-05-15 01:49:15 +0000 UTC (0+3 container statuses recorded) May 15 01:50:15.427: INFO: Container csi-provisioner ready: false, restart count 0 May 15 01:50:15.427: INFO: Container driver-registrar ready: false, restart count 0 May 15 01:50:15.427: INFO: Container mock ready: false, restart count 0 May 15 01:50:15.427: INFO: pod-e94cee9f-0eca-47df-9ce8-1d446f676d35 started at 2021-05-15 01:49:59 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.427: INFO: Container write-pod ready: false, restart count 0 May 15 01:50:15.427: INFO: pod-c3357cc4-fc25-4578-af0f-b315e3d2cb0c started at 2021-05-15 01:48:14 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.427: INFO: Container write-pod ready: false, restart count 0 May 15 01:50:15.427: INFO: pod-3f9b2ded-c6a2-495c-8705-d26384e71449 started at 2021-05-15 01:48:43 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.427: INFO: Container write-pod ready: false, restart count 0 May 15 01:50:15.427: INFO: cmk-4s6dm started at 2021-05-15 00:18:54 +0000 UTC (0+2 container statuses recorded) May 15 01:50:15.427: INFO: Container nodereport ready: true, restart count 0 May 15 01:50:15.427: INFO: Container reconcile ready: true, restart count 0 May 15 01:50:15.427: INFO: hostexec-node1-2mw4q started at 2021-05-15 01:48:56 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.427: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:50:15.427: INFO: hostexec-node1-wmdkc started at 2021-05-15 01:48:21 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.427: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:50:15.427: INFO: csi-mockplugin-attacher-0 started at 2021-05-15 01:49:15 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.427: INFO: Container csi-attacher ready: false, restart count 0 May 15 01:50:15.427: INFO: hostexec-node1-8lfk7 started at 2021-05-15 01:49:32 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.427: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:50:15.427: INFO: hostexec-node1-g9jvw started at 2021-05-15 01:49:51 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.427: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:50:15.427: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.427: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:50:15.427: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4vxcc started at 2021-05-15 00:19:00 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.427: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:50:15.427: INFO: pod-92fb967d-00bb-4449-b567-f10d76ce462f started at 2021-05-15 01:49:29 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.427: INFO: Container write-pod ready: false, restart count 0 May 15 01:50:15.427: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.427: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:50:15.427: INFO: pod-66506bc5-4a99-4e45-8c39-130b362f1eac started at 2021-05-15 01:49:14 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.427: INFO: Container write-pod ready: false, restart count 0 May 15 01:50:15.427: INFO: hostexec-node1-b5m77 started at 2021-05-15 01:48:00 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.427: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:50:15.427: INFO: pod-af39b1d5-9c8a-4ff9-938d-ae81bce79a36 started at 2021-05-15 01:48:28 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.427: INFO: Container write-pod ready: false, restart count 0 May 15 01:50:15.427: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:50:15.427: INFO: Init container install-cni ready: true, restart count 0 May 15 01:50:15.427: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:50:15.427: INFO: node-feature-discovery-worker-bw8zg started at 2021-05-15 00:18:56 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.427: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:50:15.427: INFO: collectd-mrzps started at 2021-05-15 00:19:22 +0000 UTC (0+3 container statuses recorded) May 15 01:50:15.427: INFO: Container collectd ready: true, restart count 0 May 15 01:50:15.427: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:50:15.427: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:50:15.427: INFO: hostexec-node1-8qtn9 started at 2021-05-15 01:48:34 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.427: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:50:15.427: INFO: pod-e5bfb405-29b8-435c-ab5f-54edab17f88c started at 2021-05-15 01:49:43 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.427: INFO: Container write-pod ready: false, restart count 0 W0515 01:50:15.439418 28 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:50:15.478: INFO: Latency metrics for node node1 May 15 01:50:15.478: INFO: Logging node info for node node2 May 15 01:50:15.481: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 158065 0 2021-05-14 19:58:22 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1663":"csi-mock-csi-mock-volumes-1663","csi-mock-csi-mock-volumes-3052":"csi-mock-csi-mock-volumes-3052","csi-mock-csi-mock-volumes-5678":"csi-mock-csi-mock-volumes-5678","csi-mock-csi-mock-volumes-8760":"csi-mock-csi-mock-volumes-8760","csi-mock-csi-mock-volumes-9624":"csi-mock-csi-mock-volumes-9624"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:46 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:44:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:46:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:io.kubernetes.storage.mock/node":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:10 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:10 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:10 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:50:10 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:50:15.482: INFO: Logging kubelet events for node node2 May 15 01:50:15.486: INFO: Logging pods the kubelet thinks is on node node2 May 15 01:50:15.503: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 15 01:50:15.503: INFO: Container tas-controller ready: true, restart count 0 May 15 01:50:15.503: INFO: Container tas-extender ready: true, restart count 0 May 15 01:50:15.503: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 15 01:50:15.503: INFO: Container collectd ready: true, restart count 0 May 15 01:50:15.503: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:50:15.503: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:50:15.503: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.503: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:50:15.503: INFO: pod-subpath-test-configmap-lp9m started at 2021-05-15 01:50:05 +0000 UTC (1+2 container statuses recorded) May 15 01:50:15.503: INFO: Init container init-volume-configmap-lp9m ready: false, restart count 0 May 15 01:50:15.503: INFO: Container test-container-subpath-configmap-lp9m ready: false, restart count 0 May 15 01:50:15.503: INFO: Container test-container-volume-configmap-lp9m ready: false, restart count 0 May 15 01:50:15.503: INFO: pvc-volume-tester-8prsx started at 2021-05-15 01:50:10 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.503: INFO: Container volume-tester ready: true, restart count 0 May 15 01:50:15.503: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.503: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:50:15.503: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 15 01:50:15.503: INFO: Container nodereport ready: true, restart count 0 May 15 01:50:15.503: INFO: Container reconcile ready: true, restart count 0 May 15 01:50:15.503: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.503: INFO: Container cmk-webhook ready: true, restart count 0 May 15 01:50:15.503: INFO: csi-mockplugin-0 started at 2021-05-15 01:50:02 +0000 UTC (0+3 container statuses recorded) May 15 01:50:15.503: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:50:15.503: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:50:15.503: INFO: Container mock ready: true, restart count 0 May 15 01:50:15.503: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 15 01:50:15.504: INFO: Container discover ready: false, restart count 0 May 15 01:50:15.504: INFO: Container init ready: false, restart count 0 May 15 01:50:15.504: INFO: Container install ready: false, restart count 0 May 15 01:50:15.504: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.504: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 15 01:50:15.504: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.504: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:50:15.504: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.504: INFO: Container kube-multus ready: true, restart count 1 May 15 01:50:15.504: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:50:15.504: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:50:15.504: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:50:15.504: INFO: Init container install-cni ready: true, restart count 1 May 15 01:50:15.504: INFO: Container kube-flannel ready: true, restart count 4 May 15 01:50:15.504: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:50:15.504: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:50:15.504: INFO: Container node-exporter ready: true, restart count 0 W0515 01:50:15.518119 28 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:50:15.561: INFO: Latency metrics for node node2 May 15 01:50:15.561: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-4698" for this suite. • Failure in Spec Setup (BeforeEach) [135.086 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: block] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:205 should be able to mount volume and write from pod1 [BeforeEach] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:234 May 15 01:50:14.853: Unexpected error: <*errors.errorString | 0xc003e472f0>: { s: "pod \"pod-c3357cc4-fc25-4578-af0f-b315e3d2cb0c\" is not Running: timed out waiting for the condition", } pod "pod-c3357cc4-fc25-4578-af0f-b315e3d2cb0c" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 ------------------------------ {"msg":"FAILED [sig-storage] PersistentVolumes-local [Volume type: block] One pod requesting one prebound PVC should be able to mount volume and write from pod1","total":-1,"completed":5,"skipped":686,"failed":2,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] One pod requesting one prebound PVC should be able to mount volume and read from pod1","[sig-storage] PersistentVolumes-local [Volume type: block] One pod requesting one prebound PVC should be able to mount volume and write from pod1"]} SSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:48:21.304: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: blockfswithoutformat] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes STEP: Creating block device on node "node1" using path "/tmp/local-volume-test-7bfee3da-7080-49fa-8283-14d2aaa1fe12" May 15 01:48:25.352: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-7bfee3da-7080-49fa-8283-14d2aaa1fe12 && dd if=/dev/zero of=/tmp/local-volume-test-7bfee3da-7080-49fa-8283-14d2aaa1fe12/file bs=4096 count=5120 && losetup -f /tmp/local-volume-test-7bfee3da-7080-49fa-8283-14d2aaa1fe12/file] Namespace:persistent-local-volumes-test-8821 PodName:hostexec-node1-wmdkc ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:48:25.352: INFO: >>> kubeConfig: /root/.kube/config May 15 01:48:26.288: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c E2E_LOOP_DEV=$(losetup | grep /tmp/local-volume-test-7bfee3da-7080-49fa-8283-14d2aaa1fe12/file | awk '{ print $1 }') 2>&1 > /dev/null && echo ${E2E_LOOP_DEV}] Namespace:persistent-local-volumes-test-8821 PodName:hostexec-node1-wmdkc ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:48:26.288: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:48:26.418: INFO: Creating a PV followed by a PVC May 15 01:48:26.425: INFO: Waiting for PV local-pvlkxfx to bind to PVC pvc-ztd4r May 15 01:48:26.425: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-ztd4r] to have phase Bound May 15 01:48:26.427: INFO: PersistentVolumeClaim pvc-ztd4r found but phase is Pending instead of Bound. May 15 01:48:28.430: INFO: PersistentVolumeClaim pvc-ztd4r found and phase=Bound (2.005312749s) May 15 01:48:28.430: INFO: Waiting up to 3m0s for PersistentVolume local-pvlkxfx to have phase Bound May 15 01:48:28.432: INFO: PersistentVolume local-pvlkxfx found and phase=Bound (2.093739ms) [It] should be able to write from pod1 and read from pod2 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:245 STEP: Creating pod1 to write to the PV STEP: Creating a pod May 15 01:50:28.456: FAIL: Unexpected error: <*errors.errorString | 0xc0044cc230>: { s: "pod \"pod-af39b1d5-9c8a-4ff9-938d-ae81bce79a36\" is not Running: timed out waiting for the condition", } pod "pod-af39b1d5-9c8a-4ff9-938d-ae81bce79a36" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage.twoPodsReadWriteTest(0xc002ea9760, 0xc000d34280, 0xc00398b9e0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:727 +0xaf k8s.io/kubernetes/test/e2e/storage.glob..func20.2.4.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:246 +0x45 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000703800) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc000703800) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc000703800, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [Volume type: blockfswithoutformat] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:50:28.457: INFO: Deleting PersistentVolumeClaim "pvc-ztd4r" May 15 01:50:28.461: INFO: Deleting PersistentVolume "local-pvlkxfx" May 15 01:50:28.465: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c E2E_LOOP_DEV=$(losetup | grep /tmp/local-volume-test-7bfee3da-7080-49fa-8283-14d2aaa1fe12/file | awk '{ print $1 }') 2>&1 > /dev/null && echo ${E2E_LOOP_DEV}] Namespace:persistent-local-volumes-test-8821 PodName:hostexec-node1-wmdkc ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:50:28.465: INFO: >>> kubeConfig: /root/.kube/config STEP: Tear down block device "/dev/loop5" on node "node1" at path /tmp/local-volume-test-7bfee3da-7080-49fa-8283-14d2aaa1fe12/file May 15 01:50:28.597: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c losetup -d /dev/loop5] Namespace:persistent-local-volumes-test-8821 PodName:hostexec-node1-wmdkc ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:50:28.597: INFO: >>> kubeConfig: /root/.kube/config STEP: Removing the test directory /tmp/local-volume-test-7bfee3da-7080-49fa-8283-14d2aaa1fe12 May 15 01:50:28.740: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-7bfee3da-7080-49fa-8283-14d2aaa1fe12] Namespace:persistent-local-volumes-test-8821 PodName:hostexec-node1-wmdkc ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:50:28.740: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "persistent-local-volumes-test-8821". STEP: Found 14 events. May 15 01:50:28.870: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for hostexec-node1-wmdkc: { } Scheduled: Successfully assigned persistent-local-volumes-test-8821/hostexec-node1-wmdkc to node1 May 15 01:50:28.870: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-af39b1d5-9c8a-4ff9-938d-ae81bce79a36: { } Scheduled: Successfully assigned persistent-local-volumes-test-8821/pod-af39b1d5-9c8a-4ff9-938d-ae81bce79a36 to node1 May 15 01:50:28.870: INFO: At 2021-05-15 01:48:22 +0000 UTC - event for hostexec-node1-wmdkc: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 15 01:50:28.870: INFO: At 2021-05-15 01:48:22 +0000 UTC - event for hostexec-node1-wmdkc: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 482.71151ms May 15 01:50:28.870: INFO: At 2021-05-15 01:48:22 +0000 UTC - event for hostexec-node1-wmdkc: {kubelet node1} Created: Created container agnhost-container May 15 01:50:28.870: INFO: At 2021-05-15 01:48:23 +0000 UTC - event for hostexec-node1-wmdkc: {kubelet node1} Started: Started container agnhost-container May 15 01:50:28.870: INFO: At 2021-05-15 01:48:30 +0000 UTC - event for pod-af39b1d5-9c8a-4ff9-938d-ae81bce79a36: {multus } AddedInterface: Add eth0 [10.244.3.161/24] May 15 01:50:28.870: INFO: At 2021-05-15 01:48:30 +0000 UTC - event for pod-af39b1d5-9c8a-4ff9-938d-ae81bce79a36: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:50:28.870: INFO: At 2021-05-15 01:48:31 +0000 UTC - event for pod-af39b1d5-9c8a-4ff9-938d-ae81bce79a36: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:50:28.870: INFO: At 2021-05-15 01:48:31 +0000 UTC - event for pod-af39b1d5-9c8a-4ff9-938d-ae81bce79a36: {kubelet node1} Failed: Error: ErrImagePull May 15 01:50:28.870: INFO: At 2021-05-15 01:48:31 +0000 UTC - event for pod-af39b1d5-9c8a-4ff9-938d-ae81bce79a36: {kubelet node1} SandboxChanged: Pod sandbox changed, it will be killed and re-created. May 15 01:50:28.870: INFO: At 2021-05-15 01:48:33 +0000 UTC - event for pod-af39b1d5-9c8a-4ff9-938d-ae81bce79a36: {multus } AddedInterface: Add eth0 [10.244.3.162/24] May 15 01:50:28.870: INFO: At 2021-05-15 01:48:33 +0000 UTC - event for pod-af39b1d5-9c8a-4ff9-938d-ae81bce79a36: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 15 01:50:28.870: INFO: At 2021-05-15 01:48:33 +0000 UTC - event for pod-af39b1d5-9c8a-4ff9-938d-ae81bce79a36: {kubelet node1} Failed: Error: ImagePullBackOff May 15 01:50:28.873: INFO: POD NODE PHASE GRACE CONDITIONS May 15 01:50:28.873: INFO: hostexec-node1-wmdkc node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:48:21 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:48:23 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:48:23 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:48:21 +0000 UTC }] May 15 01:50:28.873: INFO: pod-af39b1d5-9c8a-4ff9-938d-ae81bce79a36 node1 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:48:28 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:48:28 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:48:28 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:48:28 +0000 UTC }] May 15 01:50:28.873: INFO: May 15 01:50:28.877: INFO: Logging node info for node master1 May 15 01:50:28.879: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 158405 0 2021-05-14 19:56:35 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:28 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:28 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:28 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:50:28 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[@ :],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:50:28.880: INFO: Logging kubelet events for node master1 May 15 01:50:28.884: INFO: Logging pods the kubelet thinks is on node master1 May 15 01:50:28.894: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:50:28.894: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:50:28.894: INFO: kubernetes-metrics-scraper-678c97765c-fswrn started at 2021-05-15 00:18:49 +0000 UTC (0+1 container statuses recorded) May 15 01:50:28.894: INFO: Container kubernetes-metrics-scraper ready: true, restart count 0 May 15 01:50:28.894: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:50:28.894: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:50:28.894: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:50:28.894: INFO: Container kube-multus ready: true, restart count 1 May 15 01:50:28.894: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 15 01:50:28.894: INFO: Container docker-registry ready: true, restart count 0 May 15 01:50:28.894: INFO: Container nginx ready: true, restart count 0 May 15 01:50:28.894: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 15 01:50:28.894: INFO: Container nfd-controller ready: true, restart count 0 May 15 01:50:28.894: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:50:28.894: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:50:28.894: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:50:28.894: INFO: Init container install-cni ready: true, restart count 0 May 15 01:50:28.894: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:50:28.894: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:50:28.894: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:50:28.894: INFO: Container node-exporter ready: true, restart count 0 May 15 01:50:28.894: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:50:28.894: INFO: Container kube-scheduler ready: true, restart count 0 W0515 01:50:28.907598 34 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:50:28.938: INFO: Latency metrics for node master1 May 15 01:50:28.938: INFO: Logging node info for node master2 May 15 01:50:28.940: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 158382 0 2021-05-14 19:57:04 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:26 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:26 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:26 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:50:26 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:50:28.941: INFO: Logging kubelet events for node master2 May 15 01:50:28.944: INFO: Logging pods the kubelet thinks is on node master2 May 15 01:50:28.953: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:50:28.953: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:50:28.953: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:50:28.953: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:50:28.953: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:50:28.953: INFO: Container kube-scheduler ready: true, restart count 2 May 15 01:50:28.953: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:50:28.953: INFO: Init container install-cni ready: true, restart count 0 May 15 01:50:28.953: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:50:28.953: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 15 01:50:28.953: INFO: Container autoscaler ready: true, restart count 2 May 15 01:50:28.953: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:50:28.953: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:50:28.953: INFO: Container node-exporter ready: true, restart count 0 May 15 01:50:28.953: INFO: prometheus-operator-5bb8cb9d8f-fqb87 started at 2021-05-15 00:18:49 +0000 UTC (0+2 container statuses recorded) May 15 01:50:28.953: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:50:28.953: INFO: Container prometheus-operator ready: true, restart count 0 May 15 01:50:28.953: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:50:28.953: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:50:28.953: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:50:28.953: INFO: Container kube-multus ready: true, restart count 1 May 15 01:50:28.953: INFO: coredns-7677f9bb54-96w24 started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:50:28.953: INFO: Container coredns ready: true, restart count 0 W0515 01:50:28.965902 34 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:50:28.991: INFO: Latency metrics for node master2 May 15 01:50:28.991: INFO: Logging node info for node master3 May 15 01:50:28.993: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 158329 0 2021-05-14 19:57:14 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234767872 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324603904 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:24 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:24 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:24 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:50:24 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:50:28.994: INFO: Logging kubelet events for node master3 May 15 01:50:28.998: INFO: Logging pods the kubelet thinks is on node master3 May 15 01:50:29.006: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:50:29.006: INFO: Container kube-proxy ready: true, restart count 1 May 15 01:50:29.006: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:50:29.006: INFO: Init container install-cni ready: true, restart count 0 May 15 01:50:29.006: INFO: Container kube-flannel ready: true, restart count 2 May 15 01:50:29.006: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:50:29.006: INFO: Container kube-multus ready: true, restart count 1 May 15 01:50:29.006: INFO: coredns-7677f9bb54-rpj8c started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:50:29.006: INFO: Container coredns ready: true, restart count 0 May 15 01:50:29.006: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:50:29.006: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:50:29.006: INFO: Container node-exporter ready: true, restart count 0 May 15 01:50:29.006: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 15 01:50:29.006: INFO: Container kube-controller-manager ready: true, restart count 3 May 15 01:50:29.006: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:50:29.006: INFO: Container kube-scheduler ready: true, restart count 3 May 15 01:50:29.006: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:50:29.006: INFO: Container kube-apiserver ready: true, restart count 0 W0515 01:50:29.017836 34 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:50:29.043: INFO: Latency metrics for node master3 May 15 01:50:29.043: INFO: Logging node info for node node1 May 15 01:50:29.045: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 158398 0 2021-05-14 19:58:21 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1602":"csi-mock-csi-mock-volumes-1602","csi-mock-csi-mock-volumes-253":"csi-mock-csi-mock-volumes-253","csi-mock-csi-mock-volumes-7734":"csi-mock-csi-mock-volumes-7734","csi-mock-csi-mock-volumes-9728":"csi-mock-csi-mock-volumes-9728","csi-mock-csi-mock-volumes-9809":"csi-mock-csi-mock-volumes-9809"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:00 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kubelet Update v1 2021-05-15 01:49:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:50:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}},"f:status":{"f:volumesAttached":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:25 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:25 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:25 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:50:25 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[@ :],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-253^4,DevicePath:,},},Config:nil,},} May 15 01:50:29.045: INFO: Logging kubelet events for node node1 May 15 01:50:29.049: INFO: Logging pods the kubelet thinks is on node node1 May 15 01:50:29.070: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:50:29.071: INFO: Container kube-multus ready: true, restart count 1 May 15 01:50:29.071: INFO: prometheus-k8s-0 started at 2021-05-15 00:19:01 +0000 UTC (0+5 container statuses recorded) May 15 01:50:29.071: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 15 01:50:29.071: INFO: Container grafana ready: true, restart count 0 May 15 01:50:29.071: INFO: Container prometheus ready: true, restart count 26 May 15 01:50:29.071: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 15 01:50:29.071: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 15 01:50:29.071: INFO: hostexec-node1-z9htb started at 2021-05-15 01:49:24 +0000 UTC (0+1 container statuses recorded) May 15 01:50:29.071: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:50:29.071: INFO: node-exporter-flvqz started at 2021-05-15 00:18:55 +0000 UTC (0+2 container statuses recorded) May 15 01:50:29.071: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:50:29.071: INFO: Container node-exporter ready: true, restart count 0 May 15 01:50:29.071: INFO: pod-e94cee9f-0eca-47df-9ce8-1d446f676d35 started at 2021-05-15 01:49:59 +0000 UTC (0+1 container statuses recorded) May 15 01:50:29.071: INFO: Container write-pod ready: false, restart count 0 May 15 01:50:29.071: INFO: csi-mockplugin-attacher-0 started at 2021-05-15 01:50:15 +0000 UTC (0+1 container statuses recorded) May 15 01:50:29.071: INFO: Container csi-attacher ready: true, restart count 0 May 15 01:50:29.071: INFO: pod-3f9b2ded-c6a2-495c-8705-d26384e71449 started at 2021-05-15 01:48:43 +0000 UTC (0+1 container statuses recorded) May 15 01:50:29.071: INFO: Container write-pod ready: false, restart count 0 May 15 01:50:29.071: INFO: cmk-4s6dm started at 2021-05-15 00:18:54 +0000 UTC (0+2 container statuses recorded) May 15 01:50:29.071: INFO: Container nodereport ready: true, restart count 0 May 15 01:50:29.071: INFO: Container reconcile ready: true, restart count 0 May 15 01:50:29.071: INFO: hostexec-node1-2mw4q started at 2021-05-15 01:48:56 +0000 UTC (0+1 container statuses recorded) May 15 01:50:29.071: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:50:29.071: INFO: csi-mockplugin-0 started at 2021-05-15 01:50:15 +0000 UTC (0+3 container statuses recorded) May 15 01:50:29.071: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:50:29.071: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:50:29.071: INFO: Container mock ready: true, restart count 0 May 15 01:50:29.071: INFO: hostexec-node1-wmdkc started at 2021-05-15 01:48:21 +0000 UTC (0+1 container statuses recorded) May 15 01:50:29.071: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:50:29.071: INFO: pvc-volume-tester-bd8nn started at 2021-05-15 01:50:27 +0000 UTC (0+1 container statuses recorded) May 15 01:50:29.071: INFO: Container volume-tester ready: false, restart count 0 May 15 01:50:29.071: INFO: hostexec-node1-8lfk7 started at 2021-05-15 01:49:32 +0000 UTC (0+1 container statuses recorded) May 15 01:50:29.071: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:50:29.071: INFO: hostexec-node1-g9jvw started at 2021-05-15 01:49:51 +0000 UTC (0+1 container statuses recorded) May 15 01:50:29.071: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:50:29.071: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:50:29.071: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:50:29.071: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4vxcc started at 2021-05-15 00:19:00 +0000 UTC (0+1 container statuses recorded) May 15 01:50:29.071: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:50:29.071: INFO: pod-92fb967d-00bb-4449-b567-f10d76ce462f started at 2021-05-15 01:49:29 +0000 UTC (0+1 container statuses recorded) May 15 01:50:29.071: INFO: Container write-pod ready: false, restart count 0 May 15 01:50:29.071: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:50:29.071: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:50:29.071: INFO: csi-mockplugin-resizer-0 started at 2021-05-15 01:50:15 +0000 UTC (0+1 container statuses recorded) May 15 01:50:29.071: INFO: Container csi-resizer ready: true, restart count 0 May 15 01:50:29.071: INFO: pod-af39b1d5-9c8a-4ff9-938d-ae81bce79a36 started at 2021-05-15 01:48:28 +0000 UTC (0+1 container statuses recorded) May 15 01:50:29.071: INFO: Container write-pod ready: false, restart count 0 May 15 01:50:29.071: INFO: pod-66506bc5-4a99-4e45-8c39-130b362f1eac started at 2021-05-15 01:49:14 +0000 UTC (0+1 container statuses recorded) May 15 01:50:29.071: INFO: Container write-pod ready: false, restart count 0 May 15 01:50:29.071: INFO: pod-e5bfb405-29b8-435c-ab5f-54edab17f88c started at 2021-05-15 01:49:43 +0000 UTC (0+1 container statuses recorded) May 15 01:50:29.071: INFO: Container write-pod ready: false, restart count 0 May 15 01:50:29.071: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:50:29.071: INFO: Init container install-cni ready: true, restart count 0 May 15 01:50:29.071: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:50:29.071: INFO: node-feature-discovery-worker-bw8zg started at 2021-05-15 00:18:56 +0000 UTC (0+1 container statuses recorded) May 15 01:50:29.071: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:50:29.071: INFO: collectd-mrzps started at 2021-05-15 00:19:22 +0000 UTC (0+3 container statuses recorded) May 15 01:50:29.071: INFO: Container collectd ready: true, restart count 0 May 15 01:50:29.071: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:50:29.071: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:50:29.071: INFO: hostexec-node1-8qtn9 started at 2021-05-15 01:48:34 +0000 UTC (0+1 container statuses recorded) May 15 01:50:29.071: INFO: Container agnhost-container ready: true, restart count 0 W0515 01:50:29.083829 34 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:50:29.124: INFO: Latency metrics for node node1 May 15 01:50:29.124: INFO: Logging node info for node node2 May 15 01:50:29.127: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 158266 0 2021-05-14 19:58:22 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1663":"csi-mock-csi-mock-volumes-1663","csi-mock-csi-mock-volumes-3052":"csi-mock-csi-mock-volumes-3052","csi-mock-csi-mock-volumes-5678":"csi-mock-csi-mock-volumes-5678","csi-mock-csi-mock-volumes-8760":"csi-mock-csi-mock-volumes-8760","csi-mock-csi-mock-volumes-9624":"csi-mock-csi-mock-volumes-9624"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:46 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:44:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:46:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:io.kubernetes.storage.mock/node":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:20 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:20 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:20 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:50:20 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:50:29.128: INFO: Logging kubelet events for node node2 May 15 01:50:29.132: INFO: Logging pods the kubelet thinks is on node node2 May 15 01:50:29.144: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 15 01:50:29.144: INFO: Container discover ready: false, restart count 0 May 15 01:50:29.144: INFO: Container init ready: false, restart count 0 May 15 01:50:29.144: INFO: Container install ready: false, restart count 0 May 15 01:50:29.144: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:50:29.144: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:50:29.144: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:50:29.144: INFO: Container kube-multus ready: true, restart count 1 May 15 01:50:29.144: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 15 01:50:29.144: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 15 01:50:29.144: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:50:29.144: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:50:29.144: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:50:29.144: INFO: Init container install-cni ready: true, restart count 1 May 15 01:50:29.144: INFO: Container kube-flannel ready: true, restart count 4 May 15 01:50:29.144: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:50:29.144: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:50:29.144: INFO: Container node-exporter ready: true, restart count 0 May 15 01:50:29.144: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 15 01:50:29.144: INFO: Container tas-controller ready: true, restart count 0 May 15 01:50:29.144: INFO: Container tas-extender ready: true, restart count 0 May 15 01:50:29.144: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 15 01:50:29.144: INFO: Container collectd ready: true, restart count 0 May 15 01:50:29.144: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:50:29.144: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:50:29.144: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 15 01:50:29.144: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:50:29.144: INFO: pod-subpath-test-configmap-lp9m started at 2021-05-15 01:50:05 +0000 UTC (1+2 container statuses recorded) May 15 01:50:29.144: INFO: Init container init-volume-configmap-lp9m ready: false, restart count 0 May 15 01:50:29.144: INFO: Container test-container-subpath-configmap-lp9m ready: false, restart count 0 May 15 01:50:29.144: INFO: Container test-container-volume-configmap-lp9m ready: false, restart count 0 May 15 01:50:29.144: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 15 01:50:29.144: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:50:29.144: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 15 01:50:29.144: INFO: Container nodereport ready: true, restart count 0 May 15 01:50:29.144: INFO: Container reconcile ready: true, restart count 0 May 15 01:50:29.144: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 15 01:50:29.144: INFO: Container cmk-webhook ready: true, restart count 0 May 15 01:50:29.144: INFO: csi-mockplugin-0 started at 2021-05-15 01:50:02 +0000 UTC (0+3 container statuses recorded) May 15 01:50:29.144: INFO: Container csi-provisioner ready: false, restart count 0 May 15 01:50:29.144: INFO: Container driver-registrar ready: false, restart count 0 May 15 01:50:29.144: INFO: Container mock ready: false, restart count 0 W0515 01:50:29.157171 34 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:50:29.198: INFO: Latency metrics for node node2 May 15 01:50:29.198: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-8821" for this suite. • Failure [127.901 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: blockfswithoutformat] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 Two pods mounting a local volume at the same time /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:244 should be able to write from pod1 and read from pod2 [It] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:245 May 15 01:50:28.456: Unexpected error: <*errors.errorString | 0xc0044cc230>: { s: "pod \"pod-af39b1d5-9c8a-4ff9-938d-ae81bce79a36\" is not Running: timed out waiting for the condition", } pod "pod-af39b1d5-9c8a-4ff9-938d-ae81bce79a36" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:727 ------------------------------ {"msg":"FAILED [sig-storage] PersistentVolumes-local [Volume type: blockfswithoutformat] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2","total":-1,"completed":0,"skipped":71,"failed":3,"failures":["[sig-storage] PersistentVolumes-local [Volume type: blockfswithoutformat] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir-link-bindmounted] One pod requesting one prebound PVC should be able to mount volume and read from pod1","[sig-storage] PersistentVolumes-local [Volume type: blockfswithoutformat] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2"]} SSSSSSS ------------------------------ [BeforeEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:50:02.805: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename csi-mock-volumes STEP: Waiting for a default service account to be provisioned in namespace [It] exhausted, immediate binding /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:805 STEP: Building a driver namespace object, basename csi-mock-volumes-1663 STEP: Waiting for a default service account to be provisioned in namespace STEP: deploying csi mock driver May 15 01:50:02.861: INFO: creating *v1.ServiceAccount: csi-mock-volumes-1663-7538/csi-attacher May 15 01:50:02.863: INFO: creating *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-1663 May 15 01:50:02.863: INFO: Define cluster role external-attacher-runner-csi-mock-volumes-1663 May 15 01:50:02.866: INFO: creating *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-1663 May 15 01:50:02.869: INFO: creating *v1.Role: csi-mock-volumes-1663-7538/external-attacher-cfg-csi-mock-volumes-1663 May 15 01:50:02.871: INFO: creating *v1.RoleBinding: csi-mock-volumes-1663-7538/csi-attacher-role-cfg May 15 01:50:02.874: INFO: creating *v1.ServiceAccount: csi-mock-volumes-1663-7538/csi-provisioner May 15 01:50:02.876: INFO: creating *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-1663 May 15 01:50:02.876: INFO: Define cluster role external-provisioner-runner-csi-mock-volumes-1663 May 15 01:50:02.879: INFO: creating *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-1663 May 15 01:50:02.883: INFO: creating *v1.Role: csi-mock-volumes-1663-7538/external-provisioner-cfg-csi-mock-volumes-1663 May 15 01:50:02.887: INFO: creating *v1.RoleBinding: csi-mock-volumes-1663-7538/csi-provisioner-role-cfg May 15 01:50:02.892: INFO: creating *v1.ServiceAccount: csi-mock-volumes-1663-7538/csi-resizer May 15 01:50:02.897: INFO: creating *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-1663 May 15 01:50:02.897: INFO: Define cluster role external-resizer-runner-csi-mock-volumes-1663 May 15 01:50:02.902: INFO: creating *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-1663 May 15 01:50:02.904: INFO: creating *v1.Role: csi-mock-volumes-1663-7538/external-resizer-cfg-csi-mock-volumes-1663 May 15 01:50:02.924: INFO: creating *v1.RoleBinding: csi-mock-volumes-1663-7538/csi-resizer-role-cfg May 15 01:50:02.926: INFO: creating *v1.ServiceAccount: csi-mock-volumes-1663-7538/csi-mock May 15 01:50:02.929: INFO: creating *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-1663 May 15 01:50:02.931: INFO: creating *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-1663 May 15 01:50:02.934: INFO: creating *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-1663 May 15 01:50:02.937: INFO: creating *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-1663 May 15 01:50:02.939: INFO: creating *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-1663 May 15 01:50:02.942: INFO: creating *v1.StorageClass: csi-mock-sc-csi-mock-volumes-1663 May 15 01:50:02.944: INFO: creating *v1.StatefulSet: csi-mock-volumes-1663-7538/csi-mockplugin May 15 01:50:02.948: INFO: creating *v1.CSIDriver: csi-mock-csi-mock-volumes-1663 May 15 01:50:02.951: INFO: waiting up to 4m0s for CSIDriver "csi-mock-csi-mock-volumes-1663" May 15 01:50:02.952: INFO: waiting for CSIDriver csi-mock-csi-mock-volumes-1663 to register on node node2 STEP: Creating pod May 15 01:50:07.968: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil May 15 01:50:07.974: INFO: Waiting up to 5m0s for PersistentVolumeClaims [pvc-bh9ch] to have phase Bound May 15 01:50:07.978: INFO: PersistentVolumeClaim pvc-bh9ch found but phase is Pending instead of Bound. May 15 01:50:09.982: INFO: PersistentVolumeClaim pvc-bh9ch found and phase=Bound (2.007560213s) May 15 01:50:14.006: INFO: Deleting pod "pvc-volume-tester-8prsx" in namespace "csi-mock-volumes-1663" May 15 01:50:14.012: INFO: Wait up to 5m0s for pod "pvc-volume-tester-8prsx" to be fully deleted STEP: Checking PVC events May 15 01:50:19.037: INFO: PVC event ADDED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-bh9ch", GenerateName:"pvc-", Namespace:"csi-mock-volumes-1663", SelfLink:"/api/v1/namespaces/csi-mock-volumes-1663/persistentvolumeclaims/pvc-bh9ch", UID:"967f32d4-e371-401e-9719-684fcad7cbd7", ResourceVersion:"157967", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63756640207, loc:(*time.Location)(0x770c940)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"e2e.test", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc002aaa9c0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc002aaaa80)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"", StorageClassName:(*string)(0xc0007384e0), VolumeMode:(*v1.PersistentVolumeMode)(0xc000738530), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Pending", AccessModes:[]v1.PersistentVolumeAccessMode(nil), Capacity:v1.ResourceList(nil), Conditions:[]v1.PersistentVolumeClaimCondition(nil)}} May 15 01:50:19.037: INFO: PVC event MODIFIED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-bh9ch", GenerateName:"pvc-", Namespace:"csi-mock-volumes-1663", SelfLink:"/api/v1/namespaces/csi-mock-volumes-1663/persistentvolumeclaims/pvc-bh9ch", UID:"967f32d4-e371-401e-9719-684fcad7cbd7", ResourceVersion:"157968", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63756640207, loc:(*time.Location)(0x770c940)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string{"volume.beta.kubernetes.io/storage-provisioner":"csi-mock-csi-mock-volumes-1663"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"e2e.test", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc002aaada0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc002aaae40)}, v1.ManagedFieldsEntry{Manager:"kube-controller-manager", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc002aaaf00), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc002aaafa0)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"", StorageClassName:(*string)(0xc000738620), VolumeMode:(*v1.PersistentVolumeMode)(0xc000738650), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Pending", AccessModes:[]v1.PersistentVolumeAccessMode(nil), Capacity:v1.ResourceList(nil), Conditions:[]v1.PersistentVolumeClaimCondition(nil)}} May 15 01:50:19.038: INFO: PVC event MODIFIED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-bh9ch", GenerateName:"pvc-", Namespace:"csi-mock-volumes-1663", SelfLink:"/api/v1/namespaces/csi-mock-volumes-1663/persistentvolumeclaims/pvc-bh9ch", UID:"967f32d4-e371-401e-9719-684fcad7cbd7", ResourceVersion:"157977", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63756640207, loc:(*time.Location)(0x770c940)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string{"pv.kubernetes.io/bind-completed":"yes", "pv.kubernetes.io/bound-by-controller":"yes", "volume.beta.kubernetes.io/storage-provisioner":"csi-mock-csi-mock-volumes-1663"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"e2e.test", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc0014692a0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc001469300)}, v1.ManagedFieldsEntry{Manager:"kube-controller-manager", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc001469320), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0014693a0)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"pvc-967f32d4-e371-401e-9719-684fcad7cbd7", StorageClassName:(*string)(0xc001dec550), VolumeMode:(*v1.PersistentVolumeMode)(0xc001dec560), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Pending", AccessModes:[]v1.PersistentVolumeAccessMode(nil), Capacity:v1.ResourceList(nil), Conditions:[]v1.PersistentVolumeClaimCondition(nil)}} May 15 01:50:19.038: INFO: PVC event MODIFIED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-bh9ch", GenerateName:"pvc-", Namespace:"csi-mock-volumes-1663", SelfLink:"/api/v1/namespaces/csi-mock-volumes-1663/persistentvolumeclaims/pvc-bh9ch", UID:"967f32d4-e371-401e-9719-684fcad7cbd7", ResourceVersion:"157979", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63756640207, loc:(*time.Location)(0x770c940)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string{"pv.kubernetes.io/bind-completed":"yes", "pv.kubernetes.io/bound-by-controller":"yes", "volume.beta.kubernetes.io/storage-provisioner":"csi-mock-csi-mock-volumes-1663"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"e2e.test", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc002895ae0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc002895b00)}, v1.ManagedFieldsEntry{Manager:"kube-controller-manager", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc002895be0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc002895c00)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"pvc-967f32d4-e371-401e-9719-684fcad7cbd7", StorageClassName:(*string)(0xc002641350), VolumeMode:(*v1.PersistentVolumeMode)(0xc002641360), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Bound", AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Capacity:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}, Conditions:[]v1.PersistentVolumeClaimCondition(nil)}} May 15 01:50:19.038: INFO: PVC event MODIFIED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-bh9ch", GenerateName:"pvc-", Namespace:"csi-mock-volumes-1663", SelfLink:"/api/v1/namespaces/csi-mock-volumes-1663/persistentvolumeclaims/pvc-bh9ch", UID:"967f32d4-e371-401e-9719-684fcad7cbd7", ResourceVersion:"158219", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63756640207, loc:(*time.Location)(0x770c940)}}, DeletionTimestamp:(*v1.Time)(0xc002895d60), DeletionGracePeriodSeconds:(*int64)(0xc001b88538), Labels:map[string]string(nil), Annotations:map[string]string{"pv.kubernetes.io/bind-completed":"yes", "pv.kubernetes.io/bound-by-controller":"yes", "volume.beta.kubernetes.io/storage-provisioner":"csi-mock-csi-mock-volumes-1663"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"e2e.test", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc002895e00), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc002895ea0)}, v1.ManagedFieldsEntry{Manager:"kube-controller-manager", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc002895ec0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc002895f00)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"pvc-967f32d4-e371-401e-9719-684fcad7cbd7", StorageClassName:(*string)(0xc0026413a0), VolumeMode:(*v1.PersistentVolumeMode)(0xc0026413d0), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Bound", AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Capacity:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}, Conditions:[]v1.PersistentVolumeClaimCondition(nil)}} May 15 01:50:19.038: INFO: PVC event DELETED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-bh9ch", GenerateName:"pvc-", Namespace:"csi-mock-volumes-1663", SelfLink:"/api/v1/namespaces/csi-mock-volumes-1663/persistentvolumeclaims/pvc-bh9ch", UID:"967f32d4-e371-401e-9719-684fcad7cbd7", ResourceVersion:"158220", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63756640207, loc:(*time.Location)(0x770c940)}}, DeletionTimestamp:(*v1.Time)(0xc0012a6080), DeletionGracePeriodSeconds:(*int64)(0xc001b885e8), Labels:map[string]string(nil), Annotations:map[string]string{"pv.kubernetes.io/bind-completed":"yes", "pv.kubernetes.io/bound-by-controller":"yes", "volume.beta.kubernetes.io/storage-provisioner":"csi-mock-csi-mock-volumes-1663"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"e2e.test", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc0012a60a0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0012a61a0)}, v1.ManagedFieldsEntry{Manager:"kube-controller-manager", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc0012a61c0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0012a6260)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"pvc-967f32d4-e371-401e-9719-684fcad7cbd7", StorageClassName:(*string)(0xc002641410), VolumeMode:(*v1.PersistentVolumeMode)(0xc002641420), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Bound", AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Capacity:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}, Conditions:[]v1.PersistentVolumeClaimCondition(nil)}} STEP: Deleting pod pvc-volume-tester-8prsx May 15 01:50:19.038: INFO: Deleting pod "pvc-volume-tester-8prsx" in namespace "csi-mock-volumes-1663" STEP: Deleting claim pvc-bh9ch STEP: Deleting storageclass csi-mock-volumes-1663-sc STEP: Cleaning up resources STEP: deleting the test namespace: csi-mock-volumes-1663 STEP: Waiting for namespaces [csi-mock-volumes-1663] to vanish STEP: uninstalling csi mock driver May 15 01:50:25.063: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-1663-7538/csi-attacher May 15 01:50:25.068: INFO: deleting *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-1663 May 15 01:50:25.072: INFO: deleting *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-1663 May 15 01:50:25.076: INFO: deleting *v1.Role: csi-mock-volumes-1663-7538/external-attacher-cfg-csi-mock-volumes-1663 May 15 01:50:25.080: INFO: deleting *v1.RoleBinding: csi-mock-volumes-1663-7538/csi-attacher-role-cfg May 15 01:50:25.084: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-1663-7538/csi-provisioner May 15 01:50:25.089: INFO: deleting *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-1663 May 15 01:50:25.097: INFO: deleting *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-1663 May 15 01:50:25.106: INFO: deleting *v1.Role: csi-mock-volumes-1663-7538/external-provisioner-cfg-csi-mock-volumes-1663 May 15 01:50:25.115: INFO: deleting *v1.RoleBinding: csi-mock-volumes-1663-7538/csi-provisioner-role-cfg May 15 01:50:25.120: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-1663-7538/csi-resizer May 15 01:50:25.124: INFO: deleting *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-1663 May 15 01:50:25.129: INFO: deleting *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-1663 May 15 01:50:25.133: INFO: deleting *v1.Role: csi-mock-volumes-1663-7538/external-resizer-cfg-csi-mock-volumes-1663 May 15 01:50:25.137: INFO: deleting *v1.RoleBinding: csi-mock-volumes-1663-7538/csi-resizer-role-cfg May 15 01:50:25.141: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-1663-7538/csi-mock May 15 01:50:25.145: INFO: deleting *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-1663 May 15 01:50:25.150: INFO: deleting *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-1663 May 15 01:50:25.155: INFO: deleting *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-1663 May 15 01:50:25.159: INFO: deleting *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-1663 May 15 01:50:25.163: INFO: deleting *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-1663 May 15 01:50:25.167: INFO: deleting *v1.StorageClass: csi-mock-sc-csi-mock-volumes-1663 May 15 01:50:25.171: INFO: deleting *v1.StatefulSet: csi-mock-volumes-1663-7538/csi-mockplugin May 15 01:50:25.176: INFO: deleting *v1.CSIDriver: csi-mock-csi-mock-volumes-1663 STEP: deleting the driver namespace: csi-mock-volumes-1663-7538 STEP: Waiting for namespaces [csi-mock-volumes-1663-7538] to vanish [AfterEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:50:31.191: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "csi-mock-volumes-1663" for this suite. May 15 01:50:31.200: INFO: Namespace csi-mock-volumes-1663 was already deleted STEP: Destroying namespace "csi-mock-volumes-1663-7538" for this suite. May 15 01:50:31.202: INFO: Namespace csi-mock-volumes-1663-7538 was already deleted • [SLOW TEST:28.397 seconds] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 storage capacity /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:752 exhausted, immediate binding /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:805 ------------------------------ {"msg":"PASSED [sig-storage] CSI mock volume storage capacity exhausted, immediate binding","total":-1,"completed":4,"skipped":159,"failed":2,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir-link] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2"]} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:49:15.269: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename csi-mock-volumes STEP: Waiting for a default service account to be provisioned in namespace [It] should not be passed when podInfoOnMount=nil /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:358 STEP: Building a driver namespace object, basename csi-mock-volumes-1602 STEP: Waiting for a default service account to be provisioned in namespace STEP: deploying csi mock driver May 15 01:49:15.328: INFO: creating *v1.ServiceAccount: csi-mock-volumes-1602-4948/csi-attacher May 15 01:49:15.331: INFO: creating *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-1602 May 15 01:49:15.331: INFO: Define cluster role external-attacher-runner-csi-mock-volumes-1602 May 15 01:49:15.333: INFO: creating *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-1602 May 15 01:49:15.336: INFO: creating *v1.Role: csi-mock-volumes-1602-4948/external-attacher-cfg-csi-mock-volumes-1602 May 15 01:49:15.339: INFO: creating *v1.RoleBinding: csi-mock-volumes-1602-4948/csi-attacher-role-cfg May 15 01:49:15.341: INFO: creating *v1.ServiceAccount: csi-mock-volumes-1602-4948/csi-provisioner May 15 01:49:15.343: INFO: creating *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-1602 May 15 01:49:15.343: INFO: Define cluster role external-provisioner-runner-csi-mock-volumes-1602 May 15 01:49:15.346: INFO: creating *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-1602 May 15 01:49:15.349: INFO: creating *v1.Role: csi-mock-volumes-1602-4948/external-provisioner-cfg-csi-mock-volumes-1602 May 15 01:49:15.352: INFO: creating *v1.RoleBinding: csi-mock-volumes-1602-4948/csi-provisioner-role-cfg May 15 01:49:15.354: INFO: creating *v1.ServiceAccount: csi-mock-volumes-1602-4948/csi-resizer May 15 01:49:15.357: INFO: creating *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-1602 May 15 01:49:15.357: INFO: Define cluster role external-resizer-runner-csi-mock-volumes-1602 May 15 01:49:15.359: INFO: creating *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-1602 May 15 01:49:15.362: INFO: creating *v1.Role: csi-mock-volumes-1602-4948/external-resizer-cfg-csi-mock-volumes-1602 May 15 01:49:15.365: INFO: creating *v1.RoleBinding: csi-mock-volumes-1602-4948/csi-resizer-role-cfg May 15 01:49:15.367: INFO: creating *v1.ServiceAccount: csi-mock-volumes-1602-4948/csi-mock May 15 01:49:15.370: INFO: creating *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-1602 May 15 01:49:15.373: INFO: creating *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-1602 May 15 01:49:15.376: INFO: creating *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-1602 May 15 01:49:15.378: INFO: creating *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-1602 May 15 01:49:15.381: INFO: creating *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-1602 May 15 01:49:15.383: INFO: creating *v1.StorageClass: csi-mock-sc-csi-mock-volumes-1602 May 15 01:49:15.386: INFO: creating *v1.StatefulSet: csi-mock-volumes-1602-4948/csi-mockplugin May 15 01:49:15.390: INFO: creating *v1.CSIDriver: csi-mock-csi-mock-volumes-1602 May 15 01:49:15.393: INFO: creating *v1.StatefulSet: csi-mock-volumes-1602-4948/csi-mockplugin-attacher May 15 01:49:15.396: INFO: waiting up to 4m0s for CSIDriver "csi-mock-csi-mock-volumes-1602" May 15 01:49:15.399: INFO: waiting for CSIDriver csi-mock-csi-mock-volumes-1602 to register on node node1 STEP: Creating pod May 15 01:49:24.913: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil May 15 01:49:24.918: INFO: Waiting up to 5m0s for PersistentVolumeClaims [pvc-qg7hp] to have phase Bound May 15 01:49:24.920: INFO: PersistentVolumeClaim pvc-qg7hp found but phase is Pending instead of Bound. May 15 01:49:26.922: INFO: PersistentVolumeClaim pvc-qg7hp found and phase=Bound (2.004121298s) STEP: Deleting the previously created pod May 15 01:49:46.941: INFO: Deleting pod "pvc-volume-tester-2xsfm" in namespace "csi-mock-volumes-1602" May 15 01:49:46.946: INFO: Wait up to 5m0s for pod "pvc-volume-tester-2xsfm" to be fully deleted STEP: Checking CSI driver logs May 15 01:50:00.965: INFO: Found NodeUnpublishVolume: {json: {"Method":"/csi.v1.Node/NodeUnpublishVolume","Request":{"volume_id":"4","target_path":"/var/lib/kubelet/pods/2531436b-9286-4c15-ad09-f0b0d2b779ae/volumes/kubernetes.io~csi/pvc-dcad7351-1c77-4a68-a256-4893c665a9f0/mount"},"Response":{},"Error":"","FullError":null} Method:NodeUnpublishVolume Request:{VolumeContext:map[]} FullError:{Code:OK Message:} Error:} STEP: Deleting pod pvc-volume-tester-2xsfm May 15 01:50:00.965: INFO: Deleting pod "pvc-volume-tester-2xsfm" in namespace "csi-mock-volumes-1602" STEP: Deleting claim pvc-qg7hp May 15 01:50:00.979: INFO: Waiting up to 2m0s for PersistentVolume pvc-dcad7351-1c77-4a68-a256-4893c665a9f0 to get deleted May 15 01:50:00.981: INFO: PersistentVolume pvc-dcad7351-1c77-4a68-a256-4893c665a9f0 found and phase=Bound (2.292183ms) May 15 01:50:02.984: INFO: PersistentVolume pvc-dcad7351-1c77-4a68-a256-4893c665a9f0 was removed STEP: Deleting storageclass csi-mock-volumes-1602-sc STEP: Cleaning up resources STEP: deleting the test namespace: csi-mock-volumes-1602 STEP: Waiting for namespaces [csi-mock-volumes-1602] to vanish STEP: uninstalling csi mock driver May 15 01:50:09.003: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-1602-4948/csi-attacher May 15 01:50:09.009: INFO: deleting *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-1602 May 15 01:50:09.013: INFO: deleting *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-1602 May 15 01:50:09.017: INFO: deleting *v1.Role: csi-mock-volumes-1602-4948/external-attacher-cfg-csi-mock-volumes-1602 May 15 01:50:09.021: INFO: deleting *v1.RoleBinding: csi-mock-volumes-1602-4948/csi-attacher-role-cfg May 15 01:50:09.025: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-1602-4948/csi-provisioner May 15 01:50:09.030: INFO: deleting *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-1602 May 15 01:50:09.041: INFO: deleting *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-1602 May 15 01:50:09.051: INFO: deleting *v1.Role: csi-mock-volumes-1602-4948/external-provisioner-cfg-csi-mock-volumes-1602 May 15 01:50:09.057: INFO: deleting *v1.RoleBinding: csi-mock-volumes-1602-4948/csi-provisioner-role-cfg May 15 01:50:09.061: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-1602-4948/csi-resizer May 15 01:50:09.065: INFO: deleting *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-1602 May 15 01:50:09.069: INFO: deleting *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-1602 May 15 01:50:09.073: INFO: deleting *v1.Role: csi-mock-volumes-1602-4948/external-resizer-cfg-csi-mock-volumes-1602 May 15 01:50:09.077: INFO: deleting *v1.RoleBinding: csi-mock-volumes-1602-4948/csi-resizer-role-cfg May 15 01:50:09.081: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-1602-4948/csi-mock May 15 01:50:09.085: INFO: deleting *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-1602 May 15 01:50:09.090: INFO: deleting *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-1602 May 15 01:50:09.093: INFO: deleting *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-1602 May 15 01:50:09.097: INFO: deleting *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-1602 May 15 01:50:09.101: INFO: deleting *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-1602 May 15 01:50:09.105: INFO: deleting *v1.StorageClass: csi-mock-sc-csi-mock-volumes-1602 May 15 01:50:09.109: INFO: deleting *v1.StatefulSet: csi-mock-volumes-1602-4948/csi-mockplugin May 15 01:50:09.113: INFO: deleting *v1.CSIDriver: csi-mock-csi-mock-volumes-1602 May 15 01:50:09.117: INFO: deleting *v1.StatefulSet: csi-mock-volumes-1602-4948/csi-mockplugin-attacher STEP: deleting the driver namespace: csi-mock-volumes-1602-4948 STEP: Waiting for namespaces [csi-mock-volumes-1602-4948] to vanish [AfterEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:50:37.130: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "csi-mock-volumes-1602" for this suite. May 15 01:50:37.136: INFO: Namespace csi-mock-volumes-1602 was already deleted STEP: Destroying namespace "csi-mock-volumes-1602-4948" for this suite. May 15 01:50:37.138: INFO: Namespace csi-mock-volumes-1602-4948 was already deleted • [SLOW TEST:81.868 seconds] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 CSI workload information using mock driver /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:308 should not be passed when podInfoOnMount=nil /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:358 ------------------------------ {"msg":"PASSED [sig-storage] CSI mock volume CSI workload information using mock driver should not be passed when podInfoOnMount=nil","total":-1,"completed":2,"skipped":435,"failed":2,"failures":["[sig-storage] PersistentVolumes-local [Volume type: tmpfs] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2"]} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:48:34.773: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: block] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes STEP: Creating block device on node "node1" using path "/tmp/local-volume-test-e7a52fa0-4952-4761-9185-41746370f42e" May 15 01:48:38.818: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-e7a52fa0-4952-4761-9185-41746370f42e && dd if=/dev/zero of=/tmp/local-volume-test-e7a52fa0-4952-4761-9185-41746370f42e/file bs=4096 count=5120 && losetup -f /tmp/local-volume-test-e7a52fa0-4952-4761-9185-41746370f42e/file] Namespace:persistent-local-volumes-test-7274 PodName:hostexec-node1-8qtn9 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:48:38.818: INFO: >>> kubeConfig: /root/.kube/config May 15 01:48:38.957: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c E2E_LOOP_DEV=$(losetup | grep /tmp/local-volume-test-e7a52fa0-4952-4761-9185-41746370f42e/file | awk '{ print $1 }') 2>&1 > /dev/null && echo ${E2E_LOOP_DEV}] Namespace:persistent-local-volumes-test-7274 PodName:hostexec-node1-8qtn9 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:48:38.958: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:48:39.070: INFO: Creating a PV followed by a PVC May 15 01:48:39.077: INFO: Waiting for PV local-pvjrp96 to bind to PVC pvc-tchg2 May 15 01:48:39.077: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-tchg2] to have phase Bound May 15 01:48:39.079: INFO: PersistentVolumeClaim pvc-tchg2 found but phase is Pending instead of Bound. May 15 01:48:41.082: INFO: PersistentVolumeClaim pvc-tchg2 found but phase is Pending instead of Bound. May 15 01:48:43.086: INFO: PersistentVolumeClaim pvc-tchg2 found and phase=Bound (4.009273902s) May 15 01:48:43.086: INFO: Waiting up to 3m0s for PersistentVolume local-pvjrp96 to have phase Bound May 15 01:48:43.089: INFO: PersistentVolume local-pvjrp96 found and phase=Bound (2.987794ms) [It] should be able to write from pod1 and read from pod2 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:251 STEP: Creating pod1 STEP: Creating a pod May 15 01:50:43.111: FAIL: Unexpected error: <*errors.errorString | 0xc001822450>: { s: "pod \"pod-3f9b2ded-c6a2-495c-8705-d26384e71449\" is not Running: timed out waiting for the condition", } pod "pod-3f9b2ded-c6a2-495c-8705-d26384e71449" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage.twoPodsReadWriteSerialTest(0xc00195ef20, 0xc003f73580, 0xc0011ef440) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:764 +0xaf k8s.io/kubernetes/test/e2e/storage.glob..func20.2.5.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:252 +0x45 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc0026cca80) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc0026cca80) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc0026cca80, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [Volume type: block] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:50:43.113: INFO: Deleting PersistentVolumeClaim "pvc-tchg2" May 15 01:50:43.118: INFO: Deleting PersistentVolume "local-pvjrp96" May 15 01:50:43.122: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c E2E_LOOP_DEV=$(losetup | grep /tmp/local-volume-test-e7a52fa0-4952-4761-9185-41746370f42e/file | awk '{ print $1 }') 2>&1 > /dev/null && echo ${E2E_LOOP_DEV}] Namespace:persistent-local-volumes-test-7274 PodName:hostexec-node1-8qtn9 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:50:43.122: INFO: >>> kubeConfig: /root/.kube/config STEP: Tear down block device "/dev/loop0" on node "node1" at path /tmp/local-volume-test-e7a52fa0-4952-4761-9185-41746370f42e/file May 15 01:50:43.239: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c losetup -d /dev/loop0] Namespace:persistent-local-volumes-test-7274 PodName:hostexec-node1-8qtn9 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:50:43.239: INFO: >>> kubeConfig: /root/.kube/config STEP: Removing the test directory /tmp/local-volume-test-e7a52fa0-4952-4761-9185-41746370f42e May 15 01:50:43.354: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-e7a52fa0-4952-4761-9185-41746370f42e] Namespace:persistent-local-volumes-test-7274 PodName:hostexec-node1-8qtn9 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:50:43.354: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "persistent-local-volumes-test-7274". STEP: Found 17 events. May 15 01:50:43.645: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for hostexec-node1-8qtn9: { } Scheduled: Successfully assigned persistent-local-volumes-test-7274/hostexec-node1-8qtn9 to node1 May 15 01:50:43.645: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-3f9b2ded-c6a2-495c-8705-d26384e71449: { } Scheduled: Successfully assigned persistent-local-volumes-test-7274/pod-3f9b2ded-c6a2-495c-8705-d26384e71449 to node1 May 15 01:50:43.645: INFO: At 2021-05-15 01:48:35 +0000 UTC - event for hostexec-node1-8qtn9: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 15 01:50:43.645: INFO: At 2021-05-15 01:48:36 +0000 UTC - event for hostexec-node1-8qtn9: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 478.984221ms May 15 01:50:43.645: INFO: At 2021-05-15 01:48:36 +0000 UTC - event for hostexec-node1-8qtn9: {kubelet node1} Created: Created container agnhost-container May 15 01:50:43.645: INFO: At 2021-05-15 01:48:36 +0000 UTC - event for hostexec-node1-8qtn9: {kubelet node1} Started: Started container agnhost-container May 15 01:50:43.645: INFO: At 2021-05-15 01:48:39 +0000 UTC - event for pvc-tchg2: {persistentvolume-controller } ProvisioningFailed: no volume plugin matched name: kubernetes.io/no-provisioner May 15 01:50:43.645: INFO: At 2021-05-15 01:48:43 +0000 UTC - event for pod-3f9b2ded-c6a2-495c-8705-d26384e71449: {kubelet node1} SuccessfulMountVolume: MapVolume.MapPodDevice succeeded for volume "local-pvjrp96" volumeMapPath "/var/lib/kubelet/pods/05bb596f-5dee-4f5a-9cbe-a0d961fb9b95/volumeDevices/kubernetes.io~local-volume" May 15 01:50:43.645: INFO: At 2021-05-15 01:48:43 +0000 UTC - event for pod-3f9b2ded-c6a2-495c-8705-d26384e71449: {kubelet node1} SuccessfulMountVolume: MapVolume.MapPodDevice succeeded for volume "local-pvjrp96" globalMapPath "/var/lib/kubelet/plugins/kubernetes.io~local-volume/volumeDevices/local-pvjrp96" May 15 01:50:43.645: INFO: At 2021-05-15 01:48:44 +0000 UTC - event for pod-3f9b2ded-c6a2-495c-8705-d26384e71449: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:50:43.645: INFO: At 2021-05-15 01:48:44 +0000 UTC - event for pod-3f9b2ded-c6a2-495c-8705-d26384e71449: {multus } AddedInterface: Add eth0 [10.244.3.163/24] May 15 01:50:43.645: INFO: At 2021-05-15 01:48:46 +0000 UTC - event for pod-3f9b2ded-c6a2-495c-8705-d26384e71449: {kubelet node1} Failed: Error: ErrImagePull May 15 01:50:43.645: INFO: At 2021-05-15 01:48:46 +0000 UTC - event for pod-3f9b2ded-c6a2-495c-8705-d26384e71449: {kubelet node1} SandboxChanged: Pod sandbox changed, it will be killed and re-created. May 15 01:50:43.645: INFO: At 2021-05-15 01:48:46 +0000 UTC - event for pod-3f9b2ded-c6a2-495c-8705-d26384e71449: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:50:43.645: INFO: At 2021-05-15 01:48:49 +0000 UTC - event for pod-3f9b2ded-c6a2-495c-8705-d26384e71449: {multus } AddedInterface: Add eth0 [10.244.3.164/24] May 15 01:50:43.645: INFO: At 2021-05-15 01:48:49 +0000 UTC - event for pod-3f9b2ded-c6a2-495c-8705-d26384e71449: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 15 01:50:43.645: INFO: At 2021-05-15 01:48:49 +0000 UTC - event for pod-3f9b2ded-c6a2-495c-8705-d26384e71449: {kubelet node1} Failed: Error: ImagePullBackOff May 15 01:50:43.648: INFO: POD NODE PHASE GRACE CONDITIONS May 15 01:50:43.648: INFO: hostexec-node1-8qtn9 node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:48:34 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:48:37 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:48:37 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:48:34 +0000 UTC }] May 15 01:50:43.648: INFO: pod-3f9b2ded-c6a2-495c-8705-d26384e71449 node1 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:48:43 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:48:43 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:48:43 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:48:43 +0000 UTC }] May 15 01:50:43.648: INFO: May 15 01:50:43.652: INFO: Logging node info for node master1 May 15 01:50:43.655: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 158576 0 2021-05-14 19:56:35 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:38 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:38 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:38 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:50:38 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[@ :],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:50:43.656: INFO: Logging kubelet events for node master1 May 15 01:50:43.660: INFO: Logging pods the kubelet thinks is on node master1 May 15 01:50:43.671: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.671: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:50:43.671: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:50:43.671: INFO: Init container install-cni ready: true, restart count 0 May 15 01:50:43.671: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:50:43.671: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:50:43.671: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:50:43.671: INFO: Container node-exporter ready: true, restart count 0 May 15 01:50:43.671: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.671: INFO: Container kube-scheduler ready: true, restart count 0 May 15 01:50:43.671: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.671: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:50:43.671: INFO: kubernetes-metrics-scraper-678c97765c-fswrn started at 2021-05-15 00:18:49 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.671: INFO: Container kubernetes-metrics-scraper ready: true, restart count 0 May 15 01:50:43.671: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.671: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:50:43.671: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.671: INFO: Container kube-multus ready: true, restart count 1 May 15 01:50:43.671: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 15 01:50:43.671: INFO: Container docker-registry ready: true, restart count 0 May 15 01:50:43.671: INFO: Container nginx ready: true, restart count 0 May 15 01:50:43.671: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.671: INFO: Container nfd-controller ready: true, restart count 0 W0515 01:50:43.685485 30 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:50:43.719: INFO: Latency metrics for node master1 May 15 01:50:43.719: INFO: Logging node info for node master2 May 15 01:50:43.721: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 158544 0 2021-05-14 19:57:04 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:36 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:36 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:36 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:50:36 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:50:43.721: INFO: Logging kubelet events for node master2 May 15 01:50:43.724: INFO: Logging pods the kubelet thinks is on node master2 May 15 01:50:43.733: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.733: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:50:43.733: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.733: INFO: Container kube-multus ready: true, restart count 1 May 15 01:50:43.733: INFO: coredns-7677f9bb54-96w24 started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.733: INFO: Container coredns ready: true, restart count 0 May 15 01:50:43.733: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:50:43.733: INFO: Init container install-cni ready: true, restart count 0 May 15 01:50:43.733: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:50:43.733: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.733: INFO: Container autoscaler ready: true, restart count 2 May 15 01:50:43.733: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:50:43.733: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:50:43.733: INFO: Container node-exporter ready: true, restart count 0 May 15 01:50:43.733: INFO: prometheus-operator-5bb8cb9d8f-fqb87 started at 2021-05-15 00:18:49 +0000 UTC (0+2 container statuses recorded) May 15 01:50:43.733: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:50:43.733: INFO: Container prometheus-operator ready: true, restart count 0 May 15 01:50:43.733: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.733: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:50:43.733: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.733: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:50:43.733: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.733: INFO: Container kube-scheduler ready: true, restart count 2 W0515 01:50:43.744867 30 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:50:43.769: INFO: Latency metrics for node master2 May 15 01:50:43.769: INFO: Logging node info for node master3 May 15 01:50:43.771: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 158524 0 2021-05-14 19:57:14 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234767872 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324603904 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:34 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:34 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:34 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:50:34 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:50:43.771: INFO: Logging kubelet events for node master3 May 15 01:50:43.775: INFO: Logging pods the kubelet thinks is on node master3 May 15 01:50:43.783: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.783: INFO: Container kube-multus ready: true, restart count 1 May 15 01:50:43.783: INFO: coredns-7677f9bb54-rpj8c started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.783: INFO: Container coredns ready: true, restart count 0 May 15 01:50:43.783: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:50:43.783: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:50:43.783: INFO: Container node-exporter ready: true, restart count 0 May 15 01:50:43.783: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.783: INFO: Container kube-controller-manager ready: true, restart count 3 May 15 01:50:43.783: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.783: INFO: Container kube-scheduler ready: true, restart count 3 May 15 01:50:43.783: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.783: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:50:43.783: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.783: INFO: Container kube-proxy ready: true, restart count 1 May 15 01:50:43.783: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:50:43.783: INFO: Init container install-cni ready: true, restart count 0 May 15 01:50:43.783: INFO: Container kube-flannel ready: true, restart count 2 W0515 01:50:43.794009 30 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:50:43.821: INFO: Latency metrics for node master3 May 15 01:50:43.821: INFO: Logging node info for node node1 May 15 01:50:43.824: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 158536 0 2021-05-14 19:58:21 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1602":"csi-mock-csi-mock-volumes-1602","csi-mock-csi-mock-volumes-253":"csi-mock-csi-mock-volumes-253","csi-mock-csi-mock-volumes-7734":"csi-mock-csi-mock-volumes-7734","csi-mock-csi-mock-volumes-9728":"csi-mock-csi-mock-volumes-9728","csi-mock-csi-mock-volumes-9809":"csi-mock-csi-mock-volumes-9809"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:00 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:50:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}},"f:status":{"f:volumesAttached":{}}}} {kubelet Update v1 2021-05-15 01:50:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}},"f:volumesInUse":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:35 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:35 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:35 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:50:35 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[@ :],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-253^4],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-253^4,DevicePath:,},},Config:nil,},} May 15 01:50:43.825: INFO: Logging kubelet events for node node1 May 15 01:50:43.830: INFO: Logging pods the kubelet thinks is on node node1 May 15 01:50:43.905: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.905: INFO: Container kube-multus ready: true, restart count 1 May 15 01:50:43.905: INFO: prometheus-k8s-0 started at 2021-05-15 00:19:01 +0000 UTC (0+5 container statuses recorded) May 15 01:50:43.905: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 15 01:50:43.905: INFO: Container grafana ready: true, restart count 0 May 15 01:50:43.905: INFO: Container prometheus ready: true, restart count 26 May 15 01:50:43.905: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 15 01:50:43.905: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 15 01:50:43.905: INFO: hostexec-node1-z9htb started at 2021-05-15 01:49:24 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.905: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:50:43.905: INFO: node-exporter-flvqz started at 2021-05-15 00:18:55 +0000 UTC (0+2 container statuses recorded) May 15 01:50:43.905: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:50:43.905: INFO: Container node-exporter ready: true, restart count 0 May 15 01:50:43.905: INFO: pod-58f129b2-416d-48d8-93b6-2628e4cc65d5 started at 2021-05-15 01:50:35 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.905: INFO: Container write-pod ready: false, restart count 0 May 15 01:50:43.905: INFO: pod-e94cee9f-0eca-47df-9ce8-1d446f676d35 started at 2021-05-15 01:49:59 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.905: INFO: Container write-pod ready: false, restart count 0 May 15 01:50:43.905: INFO: csi-mockplugin-attacher-0 started at 2021-05-15 01:50:15 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.905: INFO: Container csi-attacher ready: true, restart count 0 May 15 01:50:43.905: INFO: pod-6fec197a-289d-4046-b7a7-00d385f2dc10 started at 2021-05-15 01:50:35 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.905: INFO: Container write-pod ready: false, restart count 0 May 15 01:50:43.905: INFO: hostexec-node1-ktptr started at 2021-05-15 01:50:37 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.905: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:50:43.905: INFO: pod-3f9b2ded-c6a2-495c-8705-d26384e71449 started at 2021-05-15 01:48:43 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.905: INFO: Container write-pod ready: false, restart count 0 May 15 01:50:43.905: INFO: cmk-4s6dm started at 2021-05-15 00:18:54 +0000 UTC (0+2 container statuses recorded) May 15 01:50:43.905: INFO: Container nodereport ready: true, restart count 0 May 15 01:50:43.905: INFO: Container reconcile ready: true, restart count 0 May 15 01:50:43.905: INFO: hostexec-node1-2mw4q started at 2021-05-15 01:48:56 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.905: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:50:43.905: INFO: csi-mockplugin-0 started at 2021-05-15 01:50:15 +0000 UTC (0+3 container statuses recorded) May 15 01:50:43.905: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:50:43.905: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:50:43.905: INFO: Container mock ready: true, restart count 0 May 15 01:50:43.905: INFO: pod-9566f4e9-ff2d-489c-94bf-709e28a35a7d started at 2021-05-15 01:50:43 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.905: INFO: Container write-pod ready: false, restart count 0 May 15 01:50:43.905: INFO: pvc-volume-tester-bd8nn started at 2021-05-15 01:50:27 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.905: INFO: Container volume-tester ready: false, restart count 0 May 15 01:50:43.905: INFO: hostexec-node1-8lfk7 started at 2021-05-15 01:49:32 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.905: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:50:43.905: INFO: hostexec-node1-g9jvw started at 2021-05-15 01:49:51 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.905: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:50:43.905: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.905: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:50:43.905: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4vxcc started at 2021-05-15 00:19:00 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.905: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:50:43.905: INFO: pod-92fb967d-00bb-4449-b567-f10d76ce462f started at 2021-05-15 01:49:29 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.905: INFO: Container write-pod ready: false, restart count 0 May 15 01:50:43.905: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.905: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:50:43.905: INFO: csi-mockplugin-resizer-0 started at 2021-05-15 01:50:15 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.905: INFO: Container csi-resizer ready: true, restart count 0 May 15 01:50:43.905: INFO: hostexec-node1-2l9b5 started at 2021-05-15 01:50:29 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.905: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:50:43.905: INFO: hostexec-node1-dwvtr started at 2021-05-15 01:50:31 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.906: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:50:43.906: INFO: pod-66506bc5-4a99-4e45-8c39-130b362f1eac started at 2021-05-15 01:49:14 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.906: INFO: Container write-pod ready: false, restart count 0 May 15 01:50:43.906: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:50:43.906: INFO: Init container install-cni ready: true, restart count 0 May 15 01:50:43.906: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:50:43.906: INFO: node-feature-discovery-worker-bw8zg started at 2021-05-15 00:18:56 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.906: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:50:43.906: INFO: collectd-mrzps started at 2021-05-15 00:19:22 +0000 UTC (0+3 container statuses recorded) May 15 01:50:43.906: INFO: Container collectd ready: true, restart count 0 May 15 01:50:43.906: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:50:43.906: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:50:43.906: INFO: hostexec-node1-8qtn9 started at 2021-05-15 01:48:34 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.906: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:50:43.906: INFO: pod-e5bfb405-29b8-435c-ab5f-54edab17f88c started at 2021-05-15 01:49:43 +0000 UTC (0+1 container statuses recorded) May 15 01:50:43.906: INFO: Container write-pod ready: false, restart count 0 W0515 01:50:43.917887 30 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:50:44.082: INFO: Latency metrics for node node1 May 15 01:50:44.082: INFO: Logging node info for node node2 May 15 01:50:44.086: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 158595 0 2021-05-14 19:58:22 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1663":"csi-mock-csi-mock-volumes-1663","csi-mock-csi-mock-volumes-3052":"csi-mock-csi-mock-volumes-3052","csi-mock-csi-mock-volumes-5678":"csi-mock-csi-mock-volumes-5678","csi-mock-csi-mock-volumes-8760":"csi-mock-csi-mock-volumes-8760","csi-mock-csi-mock-volumes-9624":"csi-mock-csi-mock-volumes-9624"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:46 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:44:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:46:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:io.kubernetes.storage.mock/node":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:40 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:40 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:50:40 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:50:40 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:50:44.087: INFO: Logging kubelet events for node node2 May 15 01:50:44.090: INFO: Logging pods the kubelet thinks is on node node2 May 15 01:50:44.101: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 15 01:50:44.101: INFO: Container cmk-webhook ready: true, restart count 0 May 15 01:50:44.101: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 15 01:50:44.101: INFO: Container discover ready: false, restart count 0 May 15 01:50:44.101: INFO: Container init ready: false, restart count 0 May 15 01:50:44.101: INFO: Container install ready: false, restart count 0 May 15 01:50:44.101: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:50:44.101: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:50:44.101: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:50:44.101: INFO: Container kube-multus ready: true, restart count 1 May 15 01:50:44.101: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 15 01:50:44.101: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 15 01:50:44.101: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:50:44.101: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:50:44.101: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:50:44.101: INFO: Init container install-cni ready: true, restart count 1 May 15 01:50:44.101: INFO: Container kube-flannel ready: true, restart count 4 May 15 01:50:44.101: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:50:44.101: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:50:44.101: INFO: Container node-exporter ready: true, restart count 0 May 15 01:50:44.101: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 15 01:50:44.101: INFO: Container tas-controller ready: true, restart count 0 May 15 01:50:44.101: INFO: Container tas-extender ready: true, restart count 0 May 15 01:50:44.101: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 15 01:50:44.101: INFO: Container collectd ready: true, restart count 0 May 15 01:50:44.101: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:50:44.101: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:50:44.101: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 15 01:50:44.101: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:50:44.101: INFO: pod-subpath-test-configmap-lp9m started at 2021-05-15 01:50:05 +0000 UTC (1+2 container statuses recorded) May 15 01:50:44.101: INFO: Init container init-volume-configmap-lp9m ready: false, restart count 0 May 15 01:50:44.101: INFO: Container test-container-subpath-configmap-lp9m ready: false, restart count 0 May 15 01:50:44.101: INFO: Container test-container-volume-configmap-lp9m ready: false, restart count 0 May 15 01:50:44.101: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 15 01:50:44.102: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:50:44.102: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 15 01:50:44.102: INFO: Container nodereport ready: true, restart count 0 May 15 01:50:44.102: INFO: Container reconcile ready: true, restart count 0 W0515 01:50:44.113392 30 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:50:44.154: INFO: Latency metrics for node node2 May 15 01:50:44.154: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-7274" for this suite. • Failure [129.390 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: block] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 Two pods mounting a local volume one after the other /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:250 should be able to write from pod1 and read from pod2 [It] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:251 May 15 01:50:43.111: Unexpected error: <*errors.errorString | 0xc001822450>: { s: "pod \"pod-3f9b2ded-c6a2-495c-8705-d26384e71449\" is not Running: timed out waiting for the condition", } pod "pod-3f9b2ded-c6a2-495c-8705-d26384e71449" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:764 ------------------------------ {"msg":"FAILED [sig-storage] PersistentVolumes-local [Volume type: block] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2","total":-1,"completed":2,"skipped":105,"failed":2,"failures":["[sig-storage] PersistentVolumes-local [Volume type: blockfswithoutformat] One pod requesting one prebound PVC should be able to mount volume and write from pod1","[sig-storage] PersistentVolumes-local [Volume type: block] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2"]} SSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PV Protection /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:50:44.198: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename pv-protection STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PV Protection /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/pv_protection.go:51 May 15 01:50:44.218: INFO: Waiting up to 30m0s for all (but 0) nodes to be schedulable STEP: Creating a PV STEP: Waiting for PV to enter phase Available May 15 01:50:44.224: INFO: Waiting up to 30s for PersistentVolume hostpath-bx6rv to have phase Available May 15 01:50:44.227: INFO: PersistentVolume hostpath-bx6rv found but phase is Pending instead of Available. May 15 01:50:45.230: INFO: PersistentVolume hostpath-bx6rv found and phase=Available (1.006402933s) STEP: Checking that PV Protection finalizer is set [It] Verify "immediate" deletion of a PV that is not bound to a PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/pv_protection.go:99 STEP: Deleting the PV May 15 01:50:45.237: INFO: Waiting up to 3m0s for PersistentVolume hostpath-bx6rv to get deleted May 15 01:50:45.239: INFO: PersistentVolume hostpath-bx6rv found and phase=Available (2.376396ms) May 15 01:50:47.244: INFO: PersistentVolume hostpath-bx6rv was removed [AfterEach] [sig-storage] PV Protection /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:50:47.244: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "pv-protection-4467" for this suite. [AfterEach] [sig-storage] PV Protection /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/pv_protection.go:92 May 15 01:50:47.253: INFO: AfterEach: Cleaning up test resources. May 15 01:50:47.253: INFO: pvc is nil May 15 01:50:47.253: INFO: Deleting PersistentVolume "hostpath-bx6rv" • ------------------------------ {"msg":"PASSED [sig-storage] PV Protection Verify \"immediate\" deletion of a PV that is not bound to a PVC","total":-1,"completed":3,"skipped":119,"failed":2,"failures":["[sig-storage] PersistentVolumes-local [Volume type: blockfswithoutformat] One pod requesting one prebound PVC should be able to mount volume and write from pod1","[sig-storage] PersistentVolumes-local [Volume type: block] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2"]} S ------------------------------ [BeforeEach] [sig-storage] ConfigMap /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:50:47.260: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename configmap STEP: Waiting for a default service account to be provisioned in namespace [It] should be consumable from pods in volume with mappings as non-root with FSGroup [LinuxOnly] [NodeFeature:FSGroup] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/configmap_volume.go:110 STEP: Creating configMap with name configmap-test-volume-map-1d5421af-df0f-4771-be6d-025e23a61b05 STEP: Creating a pod to test consume configMaps May 15 01:50:47.294: INFO: Waiting up to 5m0s for pod "pod-configmaps-98975ade-8dbe-4781-a548-57e386c8395b" in namespace "configmap-1732" to be "Succeeded or Failed" May 15 01:50:47.297: INFO: Pod "pod-configmaps-98975ade-8dbe-4781-a548-57e386c8395b": Phase="Pending", Reason="", readiness=false. Elapsed: 2.546154ms May 15 01:50:49.300: INFO: Pod "pod-configmaps-98975ade-8dbe-4781-a548-57e386c8395b": Phase="Pending", Reason="", readiness=false. Elapsed: 2.005562505s May 15 01:50:51.304: INFO: Pod "pod-configmaps-98975ade-8dbe-4781-a548-57e386c8395b": Phase="Succeeded", Reason="", readiness=false. Elapsed: 4.010458106s STEP: Saw pod success May 15 01:50:51.304: INFO: Pod "pod-configmaps-98975ade-8dbe-4781-a548-57e386c8395b" satisfied condition "Succeeded or Failed" May 15 01:50:51.307: INFO: Trying to get logs from node node2 pod pod-configmaps-98975ade-8dbe-4781-a548-57e386c8395b container configmap-volume-test: STEP: delete the pod May 15 01:50:51.323: INFO: Waiting for pod pod-configmaps-98975ade-8dbe-4781-a548-57e386c8395b to disappear May 15 01:50:51.325: INFO: Pod pod-configmaps-98975ade-8dbe-4781-a548-57e386c8395b no longer exists [AfterEach] [sig-storage] ConfigMap /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:50:51.325: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "configmap-1732" for this suite. • ------------------------------ {"msg":"PASSED [sig-storage] ConfigMap should be consumable from pods in volume with mappings as non-root with FSGroup [LinuxOnly] [NodeFeature:FSGroup]","total":-1,"completed":4,"skipped":120,"failed":2,"failures":["[sig-storage] PersistentVolumes-local [Volume type: blockfswithoutformat] One pod requesting one prebound PVC should be able to mount volume and write from pod1","[sig-storage] PersistentVolumes-local [Volume type: block] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2"]} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:48:56.100: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: dir-link-bindmounted] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes May 15 01:48:58.144: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir /tmp/local-volume-test-f018d52f-c6be-4409-9222-b2a7c819a1ba-backend && mount --bind /tmp/local-volume-test-f018d52f-c6be-4409-9222-b2a7c819a1ba-backend /tmp/local-volume-test-f018d52f-c6be-4409-9222-b2a7c819a1ba-backend && ln -s /tmp/local-volume-test-f018d52f-c6be-4409-9222-b2a7c819a1ba-backend /tmp/local-volume-test-f018d52f-c6be-4409-9222-b2a7c819a1ba] Namespace:persistent-local-volumes-test-3955 PodName:hostexec-node1-2mw4q ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:48:58.144: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:48:58.279: INFO: Creating a PV followed by a PVC May 15 01:48:58.285: INFO: Waiting for PV local-pvpghl6 to bind to PVC pvc-d22rc May 15 01:48:58.286: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-d22rc] to have phase Bound May 15 01:48:58.288: INFO: PersistentVolumeClaim pvc-d22rc found but phase is Pending instead of Bound. May 15 01:49:00.292: INFO: PersistentVolumeClaim pvc-d22rc found but phase is Pending instead of Bound. May 15 01:49:02.295: INFO: PersistentVolumeClaim pvc-d22rc found but phase is Pending instead of Bound. May 15 01:49:04.297: INFO: PersistentVolumeClaim pvc-d22rc found but phase is Pending instead of Bound. May 15 01:49:06.300: INFO: PersistentVolumeClaim pvc-d22rc found but phase is Pending instead of Bound. May 15 01:49:08.303: INFO: PersistentVolumeClaim pvc-d22rc found but phase is Pending instead of Bound. May 15 01:49:10.306: INFO: PersistentVolumeClaim pvc-d22rc found but phase is Pending instead of Bound. May 15 01:49:12.309: INFO: PersistentVolumeClaim pvc-d22rc found but phase is Pending instead of Bound. May 15 01:49:14.313: INFO: PersistentVolumeClaim pvc-d22rc found and phase=Bound (16.0271569s) May 15 01:49:14.313: INFO: Waiting up to 3m0s for PersistentVolume local-pvpghl6 to have phase Bound May 15 01:49:14.315: INFO: PersistentVolume local-pvpghl6 found and phase=Bound (1.932925ms) [It] should be able to write from pod1 and read from pod2 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:245 STEP: Creating pod1 to write to the PV STEP: Creating a pod May 15 01:51:14.338: FAIL: Unexpected error: <*errors.errorString | 0xc0004a05b0>: { s: "pod \"pod-66506bc5-4a99-4e45-8c39-130b362f1eac\" is not Running: timed out waiting for the condition", } pod "pod-66506bc5-4a99-4e45-8c39-130b362f1eac" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage.twoPodsReadWriteTest(0xc002b7eb00, 0xc000ab9200, 0xc00447e480) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:727 +0xaf k8s.io/kubernetes/test/e2e/storage.glob..func20.2.4.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:246 +0x45 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc001881380) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc001881380) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc001881380, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [Volume type: dir-link-bindmounted] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:51:14.340: INFO: Deleting PersistentVolumeClaim "pvc-d22rc" May 15 01:51:14.345: INFO: Deleting PersistentVolume "local-pvpghl6" STEP: Removing the test directory May 15 01:51:14.350: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm /tmp/local-volume-test-f018d52f-c6be-4409-9222-b2a7c819a1ba && umount /tmp/local-volume-test-f018d52f-c6be-4409-9222-b2a7c819a1ba-backend && rm -r /tmp/local-volume-test-f018d52f-c6be-4409-9222-b2a7c819a1ba-backend] Namespace:persistent-local-volumes-test-3955 PodName:hostexec-node1-2mw4q ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:51:14.350: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "persistent-local-volumes-test-3955". STEP: Found 13 events. May 15 01:51:14.491: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for hostexec-node1-2mw4q: { } Scheduled: Successfully assigned persistent-local-volumes-test-3955/hostexec-node1-2mw4q to node1 May 15 01:51:14.491: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-66506bc5-4a99-4e45-8c39-130b362f1eac: { } Scheduled: Successfully assigned persistent-local-volumes-test-3955/pod-66506bc5-4a99-4e45-8c39-130b362f1eac to node1 May 15 01:51:14.491: INFO: At 2021-05-15 01:48:56 +0000 UTC - event for hostexec-node1-2mw4q: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 15 01:51:14.491: INFO: At 2021-05-15 01:48:57 +0000 UTC - event for hostexec-node1-2mw4q: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 488.042487ms May 15 01:51:14.491: INFO: At 2021-05-15 01:48:57 +0000 UTC - event for hostexec-node1-2mw4q: {kubelet node1} Created: Created container agnhost-container May 15 01:51:14.491: INFO: At 2021-05-15 01:48:57 +0000 UTC - event for hostexec-node1-2mw4q: {kubelet node1} Started: Started container agnhost-container May 15 01:51:14.491: INFO: At 2021-05-15 01:48:58 +0000 UTC - event for pvc-d22rc: {persistentvolume-controller } ProvisioningFailed: no volume plugin matched name: kubernetes.io/no-provisioner May 15 01:51:14.491: INFO: At 2021-05-15 01:49:16 +0000 UTC - event for pod-66506bc5-4a99-4e45-8c39-130b362f1eac: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:51:14.491: INFO: At 2021-05-15 01:49:16 +0000 UTC - event for pod-66506bc5-4a99-4e45-8c39-130b362f1eac: {multus } AddedInterface: Add eth0 [10.244.3.165/24] May 15 01:51:14.491: INFO: At 2021-05-15 01:49:17 +0000 UTC - event for pod-66506bc5-4a99-4e45-8c39-130b362f1eac: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:51:14.491: INFO: At 2021-05-15 01:49:17 +0000 UTC - event for pod-66506bc5-4a99-4e45-8c39-130b362f1eac: {kubelet node1} Failed: Error: ErrImagePull May 15 01:51:14.491: INFO: At 2021-05-15 01:49:18 +0000 UTC - event for pod-66506bc5-4a99-4e45-8c39-130b362f1eac: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 15 01:51:14.491: INFO: At 2021-05-15 01:49:18 +0000 UTC - event for pod-66506bc5-4a99-4e45-8c39-130b362f1eac: {kubelet node1} Failed: Error: ImagePullBackOff May 15 01:51:14.493: INFO: POD NODE PHASE GRACE CONDITIONS May 15 01:51:14.493: INFO: hostexec-node1-2mw4q node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:48:56 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:48:58 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:48:58 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:48:56 +0000 UTC }] May 15 01:51:14.493: INFO: pod-66506bc5-4a99-4e45-8c39-130b362f1eac node1 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:49:14 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:49:14 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:49:14 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:49:14 +0000 UTC }] May 15 01:51:14.493: INFO: May 15 01:51:14.497: INFO: Logging node info for node master1 May 15 01:51:14.500: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 158931 0 2021-05-14 19:56:35 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:08 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:08 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:08 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:51:08 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[@ :],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:51:14.500: INFO: Logging kubelet events for node master1 May 15 01:51:14.505: INFO: Logging pods the kubelet thinks is on node master1 May 15 01:51:14.515: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.515: INFO: Container kube-multus ready: true, restart count 1 May 15 01:51:14.515: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 15 01:51:14.515: INFO: Container docker-registry ready: true, restart count 0 May 15 01:51:14.515: INFO: Container nginx ready: true, restart count 0 May 15 01:51:14.515: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.515: INFO: Container nfd-controller ready: true, restart count 0 May 15 01:51:14.515: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.515: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:51:14.515: INFO: kubernetes-metrics-scraper-678c97765c-fswrn started at 2021-05-15 00:18:49 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.515: INFO: Container kubernetes-metrics-scraper ready: true, restart count 0 May 15 01:51:14.515: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.515: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:51:14.515: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.515: INFO: Container kube-scheduler ready: true, restart count 0 May 15 01:51:14.515: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.515: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:51:14.515: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:51:14.515: INFO: Init container install-cni ready: true, restart count 0 May 15 01:51:14.515: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:51:14.515: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:51:14.515: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:51:14.515: INFO: Container node-exporter ready: true, restart count 0 W0515 01:51:14.528707 24 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:51:14.558: INFO: Latency metrics for node master1 May 15 01:51:14.558: INFO: Logging node info for node master2 May 15 01:51:14.561: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 158920 0 2021-05-14 19:57:04 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:06 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:06 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:06 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:51:06 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:51:14.562: INFO: Logging kubelet events for node master2 May 15 01:51:14.566: INFO: Logging pods the kubelet thinks is on node master2 May 15 01:51:14.576: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.576: INFO: Container kube-multus ready: true, restart count 1 May 15 01:51:14.576: INFO: coredns-7677f9bb54-96w24 started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.576: INFO: Container coredns ready: true, restart count 0 May 15 01:51:14.576: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.576: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:51:14.576: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.576: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:51:14.576: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.576: INFO: Container kube-scheduler ready: true, restart count 2 May 15 01:51:14.576: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:51:14.576: INFO: Init container install-cni ready: true, restart count 0 May 15 01:51:14.576: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:51:14.576: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.576: INFO: Container autoscaler ready: true, restart count 2 May 15 01:51:14.576: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:51:14.576: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:51:14.576: INFO: Container node-exporter ready: true, restart count 0 May 15 01:51:14.576: INFO: prometheus-operator-5bb8cb9d8f-fqb87 started at 2021-05-15 00:18:49 +0000 UTC (0+2 container statuses recorded) May 15 01:51:14.576: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:51:14.576: INFO: Container prometheus-operator ready: true, restart count 0 May 15 01:51:14.576: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.576: INFO: Container kube-apiserver ready: true, restart count 0 W0515 01:51:14.587916 24 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:51:14.616: INFO: Latency metrics for node master2 May 15 01:51:14.616: INFO: Logging node info for node master3 May 15 01:51:14.618: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 158905 0 2021-05-14 19:57:14 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234767872 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324603904 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:04 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:04 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:04 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:51:04 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:51:14.619: INFO: Logging kubelet events for node master3 May 15 01:51:14.624: INFO: Logging pods the kubelet thinks is on node master3 May 15 01:51:14.635: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:51:14.635: INFO: Init container install-cni ready: true, restart count 0 May 15 01:51:14.635: INFO: Container kube-flannel ready: true, restart count 2 May 15 01:51:14.635: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.635: INFO: Container kube-multus ready: true, restart count 1 May 15 01:51:14.635: INFO: coredns-7677f9bb54-rpj8c started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.635: INFO: Container coredns ready: true, restart count 0 May 15 01:51:14.635: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:51:14.635: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:51:14.635: INFO: Container node-exporter ready: true, restart count 0 May 15 01:51:14.635: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.635: INFO: Container kube-controller-manager ready: true, restart count 3 May 15 01:51:14.635: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.635: INFO: Container kube-scheduler ready: true, restart count 3 May 15 01:51:14.635: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.635: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:51:14.635: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.635: INFO: Container kube-proxy ready: true, restart count 1 W0515 01:51:14.648550 24 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:51:14.678: INFO: Latency metrics for node master3 May 15 01:51:14.678: INFO: Logging node info for node node1 May 15 01:51:14.681: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 158914 0 2021-05-14 19:58:21 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1602":"csi-mock-csi-mock-volumes-1602","csi-mock-csi-mock-volumes-253":"csi-mock-csi-mock-volumes-253","csi-mock-csi-mock-volumes-7734":"csi-mock-csi-mock-volumes-7734","csi-mock-csi-mock-volumes-9728":"csi-mock-csi-mock-volumes-9728","csi-mock-csi-mock-volumes-9809":"csi-mock-csi-mock-volumes-9809"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:00 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:50:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:51:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:05 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:05 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:05 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:51:05 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[@ :],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:51:14.682: INFO: Logging kubelet events for node node1 May 15 01:51:14.686: INFO: Logging pods the kubelet thinks is on node node1 May 15 01:51:14.713: INFO: pod-e5bfb405-29b8-435c-ab5f-54edab17f88c started at 2021-05-15 01:49:43 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.713: INFO: Container write-pod ready: false, restart count 0 May 15 01:51:14.713: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:51:14.713: INFO: Init container install-cni ready: true, restart count 0 May 15 01:51:14.713: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:51:14.713: INFO: node-feature-discovery-worker-bw8zg started at 2021-05-15 00:18:56 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.713: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:51:14.713: INFO: collectd-mrzps started at 2021-05-15 00:19:22 +0000 UTC (0+3 container statuses recorded) May 15 01:51:14.713: INFO: Container collectd ready: true, restart count 0 May 15 01:51:14.713: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:51:14.713: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:51:14.713: INFO: pod-11760848-bbb5-4ae1-af82-a7a2a4af6a9c started at 2021-05-15 01:50:59 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.713: INFO: Container write-pod ready: false, restart count 0 May 15 01:51:14.713: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.713: INFO: Container kube-multus ready: true, restart count 1 May 15 01:51:14.713: INFO: prometheus-k8s-0 started at 2021-05-15 00:19:01 +0000 UTC (0+5 container statuses recorded) May 15 01:51:14.713: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 15 01:51:14.713: INFO: Container grafana ready: true, restart count 0 May 15 01:51:14.713: INFO: Container prometheus ready: true, restart count 26 May 15 01:51:14.713: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 15 01:51:14.713: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 15 01:51:14.713: INFO: hostexec-node1-z9htb started at 2021-05-15 01:49:24 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.713: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:51:14.713: INFO: hostexec-node1-ktptr started at 2021-05-15 01:50:37 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.713: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:51:14.713: INFO: node-exporter-flvqz started at 2021-05-15 00:18:55 +0000 UTC (0+2 container statuses recorded) May 15 01:51:14.713: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:51:14.713: INFO: Container node-exporter ready: true, restart count 0 May 15 01:51:14.713: INFO: pod-58f129b2-416d-48d8-93b6-2628e4cc65d5 started at 2021-05-15 01:50:35 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.713: INFO: Container write-pod ready: false, restart count 0 May 15 01:51:14.713: INFO: pod-e94cee9f-0eca-47df-9ce8-1d446f676d35 started at 2021-05-15 01:49:59 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.713: INFO: Container write-pod ready: false, restart count 0 May 15 01:51:14.713: INFO: csi-mockplugin-attacher-0 started at 2021-05-15 01:50:15 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.713: INFO: Container csi-attacher ready: true, restart count 0 May 15 01:51:14.713: INFO: pod-6fec197a-289d-4046-b7a7-00d385f2dc10 started at 2021-05-15 01:50:35 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.714: INFO: Container write-pod ready: false, restart count 0 May 15 01:51:14.714: INFO: cmk-4s6dm started at 2021-05-15 00:18:54 +0000 UTC (0+2 container statuses recorded) May 15 01:51:14.714: INFO: Container nodereport ready: true, restart count 0 May 15 01:51:14.714: INFO: Container reconcile ready: true, restart count 0 May 15 01:51:14.714: INFO: hostexec-node1-2mw4q started at 2021-05-15 01:48:56 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.714: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:51:14.714: INFO: csi-mockplugin-0 started at 2021-05-15 01:50:15 +0000 UTC (0+3 container statuses recorded) May 15 01:51:14.714: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:51:14.714: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:51:14.714: INFO: Container mock ready: true, restart count 0 May 15 01:51:14.714: INFO: pod-9566f4e9-ff2d-489c-94bf-709e28a35a7d started at 2021-05-15 01:50:43 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.714: INFO: Container write-pod ready: false, restart count 0 May 15 01:51:14.714: INFO: hostexec-node1-8lfk7 started at 2021-05-15 01:49:32 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.714: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:51:14.714: INFO: hostexec-node1-g9jvw started at 2021-05-15 01:49:51 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.714: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:51:14.714: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.714: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:51:14.714: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4vxcc started at 2021-05-15 00:19:00 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.714: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:51:14.714: INFO: pod-92fb967d-00bb-4449-b567-f10d76ce462f started at 2021-05-15 01:49:29 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.714: INFO: Container write-pod ready: false, restart count 0 May 15 01:51:14.714: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.714: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:51:14.714: INFO: csi-mockplugin-resizer-0 started at 2021-05-15 01:50:15 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.714: INFO: Container csi-resizer ready: true, restart count 0 May 15 01:51:14.714: INFO: hostexec-node1-2l9b5 started at 2021-05-15 01:50:29 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.714: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:51:14.714: INFO: hostexec-node1-dwvtr started at 2021-05-15 01:50:31 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.714: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:51:14.714: INFO: pod-66506bc5-4a99-4e45-8c39-130b362f1eac started at 2021-05-15 01:49:14 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.714: INFO: Container write-pod ready: false, restart count 0 May 15 01:51:14.714: INFO: hostexec-node1-zbxmq started at 2021-05-15 01:50:51 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.714: INFO: Container agnhost-container ready: true, restart count 0 W0515 01:51:14.726419 24 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:51:14.767: INFO: Latency metrics for node node1 May 15 01:51:14.767: INFO: Logging node info for node node2 May 15 01:51:14.771: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 158939 0 2021-05-14 19:58:22 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1663":"csi-mock-csi-mock-volumes-1663","csi-mock-csi-mock-volumes-3052":"csi-mock-csi-mock-volumes-3052","csi-mock-csi-mock-volumes-5678":"csi-mock-csi-mock-volumes-5678","csi-mock-csi-mock-volumes-8760":"csi-mock-csi-mock-volumes-8760","csi-mock-csi-mock-volumes-9624":"csi-mock-csi-mock-volumes-9624"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:46 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:44:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:46:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:io.kubernetes.storage.mock/node":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:10 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:10 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:10 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:51:10 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:51:14.772: INFO: Logging kubelet events for node node2 May 15 01:51:14.775: INFO: Logging pods the kubelet thinks is on node node2 May 15 01:51:14.786: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:51:14.786: INFO: Init container install-cni ready: true, restart count 1 May 15 01:51:14.786: INFO: Container kube-flannel ready: true, restart count 4 May 15 01:51:14.786: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:51:14.786: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:51:14.786: INFO: Container node-exporter ready: true, restart count 0 May 15 01:51:14.786: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 15 01:51:14.786: INFO: Container tas-controller ready: true, restart count 0 May 15 01:51:14.786: INFO: Container tas-extender ready: true, restart count 0 May 15 01:51:14.786: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 15 01:51:14.786: INFO: Container collectd ready: true, restart count 0 May 15 01:51:14.786: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:51:14.786: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:51:14.786: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.786: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:51:14.786: INFO: pod-subpath-test-configmap-lp9m started at 2021-05-15 01:50:05 +0000 UTC (1+2 container statuses recorded) May 15 01:51:14.786: INFO: Init container init-volume-configmap-lp9m ready: false, restart count 0 May 15 01:51:14.786: INFO: Container test-container-subpath-configmap-lp9m ready: false, restart count 0 May 15 01:51:14.786: INFO: Container test-container-volume-configmap-lp9m ready: false, restart count 0 May 15 01:51:14.786: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.786: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:51:14.786: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 15 01:51:14.786: INFO: Container nodereport ready: true, restart count 0 May 15 01:51:14.786: INFO: Container reconcile ready: true, restart count 0 May 15 01:51:14.786: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.786: INFO: Container cmk-webhook ready: true, restart count 0 May 15 01:51:14.786: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 15 01:51:14.786: INFO: Container discover ready: false, restart count 0 May 15 01:51:14.786: INFO: Container init ready: false, restart count 0 May 15 01:51:14.786: INFO: Container install ready: false, restart count 0 May 15 01:51:14.786: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.786: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:51:14.786: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.786: INFO: Container kube-multus ready: true, restart count 1 May 15 01:51:14.786: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.786: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 15 01:51:14.786: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:51:14.786: INFO: Container nginx-proxy ready: true, restart count 2 W0515 01:51:14.798547 24 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:51:14.837: INFO: Latency metrics for node node2 May 15 01:51:14.837: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-3955" for this suite. • Failure [138.747 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: dir-link-bindmounted] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 Two pods mounting a local volume at the same time /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:244 should be able to write from pod1 and read from pod2 [It] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:245 May 15 01:51:14.338: Unexpected error: <*errors.errorString | 0xc0004a05b0>: { s: "pod \"pod-66506bc5-4a99-4e45-8c39-130b362f1eac\" is not Running: timed out waiting for the condition", } pod "pod-66506bc5-4a99-4e45-8c39-130b362f1eac" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:727 ------------------------------ {"msg":"FAILED [sig-storage] PersistentVolumes-local [Volume type: dir-link-bindmounted] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2","total":-1,"completed":5,"skipped":762,"failed":3,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir] One pod requesting one prebound PVC should be able to mount volume and write from pod1","[sig-storage] PersistentVolumes-local [Volume type: tmpfs] One pod requesting one prebound PVC should be able to mount volume and read from pod1","[sig-storage] PersistentVolumes-local [Volume type: dir-link-bindmounted] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2"]} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:49:24.842: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: tmpfs] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes STEP: Creating tmpfs mount point on node "node1" at path "/tmp/local-volume-test-4f7b8616-c30d-4d5f-864e-cf44b9b7d28a" May 15 01:49:26.896: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p "/tmp/local-volume-test-4f7b8616-c30d-4d5f-864e-cf44b9b7d28a" && mount -t tmpfs -o size=10m tmpfs-"/tmp/local-volume-test-4f7b8616-c30d-4d5f-864e-cf44b9b7d28a" "/tmp/local-volume-test-4f7b8616-c30d-4d5f-864e-cf44b9b7d28a"] Namespace:persistent-local-volumes-test-2789 PodName:hostexec-node1-z9htb ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:49:26.896: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:49:27.077: INFO: Creating a PV followed by a PVC May 15 01:49:27.084: INFO: Waiting for PV local-pv7bxv5 to bind to PVC pvc-dbfvm May 15 01:49:27.084: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-dbfvm] to have phase Bound May 15 01:49:27.086: INFO: PersistentVolumeClaim pvc-dbfvm found but phase is Pending instead of Bound. May 15 01:49:29.090: INFO: PersistentVolumeClaim pvc-dbfvm found and phase=Bound (2.006036214s) May 15 01:49:29.090: INFO: Waiting up to 3m0s for PersistentVolume local-pv7bxv5 to have phase Bound May 15 01:49:29.093: INFO: PersistentVolume local-pv7bxv5 found and phase=Bound (2.878837ms) [It] should be able to write from pod1 and read from pod2 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:245 STEP: Creating pod1 to write to the PV STEP: Creating a pod May 15 01:51:29.116: FAIL: Unexpected error: <*errors.errorString | 0xc001810590>: { s: "pod \"pod-92fb967d-00bb-4449-b567-f10d76ce462f\" is not Running: timed out waiting for the condition", } pod "pod-92fb967d-00bb-4449-b567-f10d76ce462f" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage.twoPodsReadWriteTest(0xc002674580, 0xc001710480, 0xc002416060) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:727 +0xaf k8s.io/kubernetes/test/e2e/storage.glob..func20.2.4.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:246 +0x45 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc001d81980) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc001d81980) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc001d81980, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [Volume type: tmpfs] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:51:29.118: INFO: Deleting PersistentVolumeClaim "pvc-dbfvm" May 15 01:51:29.122: INFO: Deleting PersistentVolume "local-pv7bxv5" STEP: Unmount tmpfs mount point on node "node1" at path "/tmp/local-volume-test-4f7b8616-c30d-4d5f-864e-cf44b9b7d28a" May 15 01:51:29.126: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c umount "/tmp/local-volume-test-4f7b8616-c30d-4d5f-864e-cf44b9b7d28a"] Namespace:persistent-local-volumes-test-2789 PodName:hostexec-node1-z9htb ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:51:29.126: INFO: >>> kubeConfig: /root/.kube/config STEP: Removing the test directory May 15 01:51:29.251: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-4f7b8616-c30d-4d5f-864e-cf44b9b7d28a] Namespace:persistent-local-volumes-test-2789 PodName:hostexec-node1-z9htb ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:51:29.251: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "persistent-local-volumes-test-2789". STEP: Found 14 events. May 15 01:51:29.375: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for hostexec-node1-z9htb: { } Scheduled: Successfully assigned persistent-local-volumes-test-2789/hostexec-node1-z9htb to node1 May 15 01:51:29.375: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-92fb967d-00bb-4449-b567-f10d76ce462f: { } Scheduled: Successfully assigned persistent-local-volumes-test-2789/pod-92fb967d-00bb-4449-b567-f10d76ce462f to node1 May 15 01:51:29.375: INFO: At 2021-05-15 01:49:25 +0000 UTC - event for hostexec-node1-z9htb: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 15 01:51:29.375: INFO: At 2021-05-15 01:49:26 +0000 UTC - event for hostexec-node1-z9htb: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 573.160289ms May 15 01:51:29.375: INFO: At 2021-05-15 01:49:26 +0000 UTC - event for hostexec-node1-z9htb: {kubelet node1} Created: Created container agnhost-container May 15 01:51:29.375: INFO: At 2021-05-15 01:49:26 +0000 UTC - event for hostexec-node1-z9htb: {kubelet node1} Started: Started container agnhost-container May 15 01:51:29.375: INFO: At 2021-05-15 01:49:30 +0000 UTC - event for pod-92fb967d-00bb-4449-b567-f10d76ce462f: {multus } AddedInterface: Add eth0 [10.244.3.168/24] May 15 01:51:29.375: INFO: At 2021-05-15 01:49:30 +0000 UTC - event for pod-92fb967d-00bb-4449-b567-f10d76ce462f: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:51:29.375: INFO: At 2021-05-15 01:49:31 +0000 UTC - event for pod-92fb967d-00bb-4449-b567-f10d76ce462f: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:51:29.375: INFO: At 2021-05-15 01:49:31 +0000 UTC - event for pod-92fb967d-00bb-4449-b567-f10d76ce462f: {kubelet node1} Failed: Error: ErrImagePull May 15 01:51:29.375: INFO: At 2021-05-15 01:49:32 +0000 UTC - event for pod-92fb967d-00bb-4449-b567-f10d76ce462f: {kubelet node1} SandboxChanged: Pod sandbox changed, it will be killed and re-created. May 15 01:51:29.375: INFO: At 2021-05-15 01:49:34 +0000 UTC - event for pod-92fb967d-00bb-4449-b567-f10d76ce462f: {multus } AddedInterface: Add eth0 [10.244.3.169/24] May 15 01:51:29.375: INFO: At 2021-05-15 01:49:34 +0000 UTC - event for pod-92fb967d-00bb-4449-b567-f10d76ce462f: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 15 01:51:29.375: INFO: At 2021-05-15 01:49:34 +0000 UTC - event for pod-92fb967d-00bb-4449-b567-f10d76ce462f: {kubelet node1} Failed: Error: ImagePullBackOff May 15 01:51:29.378: INFO: POD NODE PHASE GRACE CONDITIONS May 15 01:51:29.378: INFO: hostexec-node1-z9htb node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:49:24 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:49:26 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:49:26 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:49:24 +0000 UTC }] May 15 01:51:29.378: INFO: pod-92fb967d-00bb-4449-b567-f10d76ce462f node1 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:49:29 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:49:29 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:49:29 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:49:29 +0000 UTC }] May 15 01:51:29.378: INFO: May 15 01:51:29.382: INFO: Logging node info for node master1 May 15 01:51:29.385: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 159267 0 2021-05-14 19:56:35 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:28 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:28 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:28 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:51:28 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[@ :],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:51:29.385: INFO: Logging kubelet events for node master1 May 15 01:51:29.390: INFO: Logging pods the kubelet thinks is on node master1 May 15 01:51:29.398: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.398: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:51:29.399: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:51:29.399: INFO: Init container install-cni ready: true, restart count 0 May 15 01:51:29.399: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:51:29.399: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:51:29.399: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:51:29.399: INFO: Container node-exporter ready: true, restart count 0 May 15 01:51:29.399: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.399: INFO: Container kube-scheduler ready: true, restart count 0 May 15 01:51:29.399: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.399: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:51:29.399: INFO: kubernetes-metrics-scraper-678c97765c-fswrn started at 2021-05-15 00:18:49 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.399: INFO: Container kubernetes-metrics-scraper ready: true, restart count 0 May 15 01:51:29.399: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.399: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:51:29.399: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.399: INFO: Container kube-multus ready: true, restart count 1 May 15 01:51:29.399: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 15 01:51:29.399: INFO: Container docker-registry ready: true, restart count 0 May 15 01:51:29.399: INFO: Container nginx ready: true, restart count 0 May 15 01:51:29.399: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.399: INFO: Container nfd-controller ready: true, restart count 0 W0515 01:51:29.411666 21 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:51:29.444: INFO: Latency metrics for node master1 May 15 01:51:29.444: INFO: Logging node info for node master2 May 15 01:51:29.447: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 159249 0 2021-05-14 19:57:04 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:26 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:26 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:26 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:51:26 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:51:29.447: INFO: Logging kubelet events for node master2 May 15 01:51:29.451: INFO: Logging pods the kubelet thinks is on node master2 May 15 01:51:29.459: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.459: INFO: Container kube-scheduler ready: true, restart count 2 May 15 01:51:29.459: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:51:29.459: INFO: Init container install-cni ready: true, restart count 0 May 15 01:51:29.459: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:51:29.459: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.459: INFO: Container autoscaler ready: true, restart count 2 May 15 01:51:29.459: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:51:29.459: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:51:29.459: INFO: Container node-exporter ready: true, restart count 0 May 15 01:51:29.459: INFO: prometheus-operator-5bb8cb9d8f-fqb87 started at 2021-05-15 00:18:49 +0000 UTC (0+2 container statuses recorded) May 15 01:51:29.459: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:51:29.459: INFO: Container prometheus-operator ready: true, restart count 0 May 15 01:51:29.459: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.459: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:51:29.459: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.459: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:51:29.459: INFO: coredns-7677f9bb54-96w24 started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.459: INFO: Container coredns ready: true, restart count 0 May 15 01:51:29.459: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.459: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:51:29.459: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.459: INFO: Container kube-multus ready: true, restart count 1 W0515 01:51:29.471421 21 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:51:29.495: INFO: Latency metrics for node master2 May 15 01:51:29.495: INFO: Logging node info for node master3 May 15 01:51:29.497: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 159234 0 2021-05-14 19:57:14 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234767872 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324603904 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:24 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:24 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:24 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:51:24 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:51:29.497: INFO: Logging kubelet events for node master3 May 15 01:51:29.501: INFO: Logging pods the kubelet thinks is on node master3 May 15 01:51:29.508: INFO: coredns-7677f9bb54-rpj8c started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.508: INFO: Container coredns ready: true, restart count 0 May 15 01:51:29.508: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:51:29.508: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:51:29.508: INFO: Container node-exporter ready: true, restart count 0 May 15 01:51:29.508: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.508: INFO: Container kube-controller-manager ready: true, restart count 3 May 15 01:51:29.508: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.508: INFO: Container kube-scheduler ready: true, restart count 3 May 15 01:51:29.508: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.508: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:51:29.508: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.508: INFO: Container kube-proxy ready: true, restart count 1 May 15 01:51:29.508: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:51:29.508: INFO: Init container install-cni ready: true, restart count 0 May 15 01:51:29.508: INFO: Container kube-flannel ready: true, restart count 2 May 15 01:51:29.508: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.508: INFO: Container kube-multus ready: true, restart count 1 W0515 01:51:29.521538 21 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:51:29.548: INFO: Latency metrics for node master3 May 15 01:51:29.548: INFO: Logging node info for node node1 May 15 01:51:29.551: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 159245 0 2021-05-14 19:58:21 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1602":"csi-mock-csi-mock-volumes-1602","csi-mock-csi-mock-volumes-253":"csi-mock-csi-mock-volumes-253","csi-mock-csi-mock-volumes-7734":"csi-mock-csi-mock-volumes-7734","csi-mock-csi-mock-volumes-895":"csi-mock-csi-mock-volumes-895","csi-mock-csi-mock-volumes-9728":"csi-mock-csi-mock-volumes-9728","csi-mock-csi-mock-volumes-9809":"csi-mock-csi-mock-volumes-9809"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:00 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:50:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:51:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:26 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:26 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:26 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:51:26 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[@ :],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:51:29.551: INFO: Logging kubelet events for node node1 May 15 01:51:29.556: INFO: Logging pods the kubelet thinks is on node node1 May 15 01:51:29.575: INFO: hostexec-node1-z9htb started at 2021-05-15 01:49:24 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.575: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:51:29.575: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.575: INFO: Container kube-multus ready: true, restart count 1 May 15 01:51:29.575: INFO: prometheus-k8s-0 started at 2021-05-15 00:19:01 +0000 UTC (0+5 container statuses recorded) May 15 01:51:29.575: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 15 01:51:29.575: INFO: Container grafana ready: true, restart count 0 May 15 01:51:29.575: INFO: Container prometheus ready: true, restart count 26 May 15 01:51:29.575: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 15 01:51:29.575: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 15 01:51:29.575: INFO: csi-mockplugin-attacher-0 started at 2021-05-15 01:50:15 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.575: INFO: Container csi-attacher ready: false, restart count 0 May 15 01:51:29.575: INFO: pod-6fec197a-289d-4046-b7a7-00d385f2dc10 started at 2021-05-15 01:50:35 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.575: INFO: Container write-pod ready: false, restart count 0 May 15 01:51:29.575: INFO: hostexec-node1-ktptr started at 2021-05-15 01:50:37 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.575: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:51:29.575: INFO: csi-mockplugin-0 started at 2021-05-15 01:51:15 +0000 UTC (0+3 container statuses recorded) May 15 01:51:29.575: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:51:29.575: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:51:29.575: INFO: Container mock ready: true, restart count 0 May 15 01:51:29.575: INFO: pvc-volume-tester-9b8r9 started at 2021-05-15 01:51:21 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.575: INFO: Container volume-tester ready: false, restart count 0 May 15 01:51:29.575: INFO: node-exporter-flvqz started at 2021-05-15 00:18:55 +0000 UTC (0+2 container statuses recorded) May 15 01:51:29.575: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:51:29.575: INFO: Container node-exporter ready: true, restart count 0 May 15 01:51:29.575: INFO: pod-58f129b2-416d-48d8-93b6-2628e4cc65d5 started at 2021-05-15 01:50:35 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.575: INFO: Container write-pod ready: false, restart count 0 May 15 01:51:29.575: INFO: pod-e94cee9f-0eca-47df-9ce8-1d446f676d35 started at 2021-05-15 01:49:59 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.575: INFO: Container write-pod ready: false, restart count 0 May 15 01:51:29.575: INFO: csi-mockplugin-0 started at 2021-05-15 01:50:15 +0000 UTC (0+3 container statuses recorded) May 15 01:51:29.575: INFO: Container csi-provisioner ready: false, restart count 0 May 15 01:51:29.575: INFO: Container driver-registrar ready: false, restart count 0 May 15 01:51:29.576: INFO: Container mock ready: false, restart count 0 May 15 01:51:29.576: INFO: pod-9566f4e9-ff2d-489c-94bf-709e28a35a7d started at 2021-05-15 01:50:43 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.576: INFO: Container write-pod ready: false, restart count 0 May 15 01:51:29.576: INFO: cmk-4s6dm started at 2021-05-15 00:18:54 +0000 UTC (0+2 container statuses recorded) May 15 01:51:29.576: INFO: Container nodereport ready: true, restart count 0 May 15 01:51:29.576: INFO: Container reconcile ready: true, restart count 0 May 15 01:51:29.576: INFO: hostexec-node1-g9jvw started at 2021-05-15 01:49:51 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.576: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:51:29.576: INFO: hostexec-node1-8lfk7 started at 2021-05-15 01:49:32 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.576: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:51:29.576: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.576: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:51:29.576: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4vxcc started at 2021-05-15 00:19:00 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.576: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:51:29.576: INFO: pod-92fb967d-00bb-4449-b567-f10d76ce462f started at 2021-05-15 01:49:29 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.576: INFO: Container write-pod ready: false, restart count 0 May 15 01:51:29.576: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.576: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:51:29.576: INFO: csi-mockplugin-resizer-0 started at 2021-05-15 01:50:15 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.576: INFO: Container csi-resizer ready: false, restart count 0 May 15 01:51:29.576: INFO: hostexec-node1-2l9b5 started at 2021-05-15 01:50:29 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.576: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:51:29.576: INFO: hostexec-node1-dwvtr started at 2021-05-15 01:50:31 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.576: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:51:29.576: INFO: hostexec-node1-zbxmq started at 2021-05-15 01:50:51 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.576: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:51:29.576: INFO: pod-11760848-bbb5-4ae1-af82-a7a2a4af6a9c started at 2021-05-15 01:50:59 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.576: INFO: Container write-pod ready: false, restart count 0 May 15 01:51:29.576: INFO: pod-e5bfb405-29b8-435c-ab5f-54edab17f88c started at 2021-05-15 01:49:43 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.576: INFO: Container write-pod ready: false, restart count 0 May 15 01:51:29.576: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:51:29.576: INFO: Init container install-cni ready: true, restart count 0 May 15 01:51:29.576: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:51:29.576: INFO: node-feature-discovery-worker-bw8zg started at 2021-05-15 00:18:56 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.576: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:51:29.576: INFO: collectd-mrzps started at 2021-05-15 00:19:22 +0000 UTC (0+3 container statuses recorded) May 15 01:51:29.576: INFO: Container collectd ready: true, restart count 0 May 15 01:51:29.576: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:51:29.576: INFO: Container rbac-proxy ready: true, restart count 0 W0515 01:51:29.586416 21 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:51:29.625: INFO: Latency metrics for node node1 May 15 01:51:29.626: INFO: Logging node info for node node2 May 15 01:51:29.628: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 159176 0 2021-05-14 19:58:22 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1663":"csi-mock-csi-mock-volumes-1663","csi-mock-csi-mock-volumes-3052":"csi-mock-csi-mock-volumes-3052","csi-mock-csi-mock-volumes-5678":"csi-mock-csi-mock-volumes-5678","csi-mock-csi-mock-volumes-8760":"csi-mock-csi-mock-volumes-8760","csi-mock-csi-mock-volumes-9624":"csi-mock-csi-mock-volumes-9624"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:46 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:44:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:46:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:io.kubernetes.storage.mock/node":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:20 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:20 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:20 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:51:20 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:51:29.629: INFO: Logging kubelet events for node node2 May 15 01:51:29.632: INFO: Logging pods the kubelet thinks is on node node2 May 15 01:51:29.653: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:51:29.653: INFO: Init container install-cni ready: true, restart count 1 May 15 01:51:29.653: INFO: Container kube-flannel ready: true, restart count 4 May 15 01:51:29.653: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:51:29.653: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:51:29.653: INFO: Container node-exporter ready: true, restart count 0 May 15 01:51:29.653: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 15 01:51:29.653: INFO: Container tas-controller ready: true, restart count 0 May 15 01:51:29.653: INFO: Container tas-extender ready: true, restart count 0 May 15 01:51:29.653: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 15 01:51:29.653: INFO: Container collectd ready: true, restart count 0 May 15 01:51:29.653: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:51:29.653: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:51:29.653: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.653: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:51:29.654: INFO: pod-subpath-test-configmap-lp9m started at 2021-05-15 01:50:05 +0000 UTC (1+2 container statuses recorded) May 15 01:51:29.654: INFO: Init container init-volume-configmap-lp9m ready: false, restart count 0 May 15 01:51:29.654: INFO: Container test-container-subpath-configmap-lp9m ready: false, restart count 0 May 15 01:51:29.654: INFO: Container test-container-volume-configmap-lp9m ready: false, restart count 0 May 15 01:51:29.654: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.654: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:51:29.654: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 15 01:51:29.654: INFO: Container nodereport ready: true, restart count 0 May 15 01:51:29.654: INFO: Container reconcile ready: true, restart count 0 May 15 01:51:29.654: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.654: INFO: Container cmk-webhook ready: true, restart count 0 May 15 01:51:29.654: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 15 01:51:29.654: INFO: Container discover ready: false, restart count 0 May 15 01:51:29.654: INFO: Container init ready: false, restart count 0 May 15 01:51:29.654: INFO: Container install ready: false, restart count 0 May 15 01:51:29.654: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.654: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:51:29.654: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.654: INFO: Container kube-multus ready: true, restart count 1 May 15 01:51:29.654: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.654: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 15 01:51:29.654: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:51:29.654: INFO: Container nginx-proxy ready: true, restart count 2 W0515 01:51:29.666787 21 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:51:29.704: INFO: Latency metrics for node node2 May 15 01:51:29.704: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-2789" for this suite. • Failure [124.874 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: tmpfs] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 Two pods mounting a local volume at the same time /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:244 should be able to write from pod1 and read from pod2 [It] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:245 May 15 01:51:29.116: Unexpected error: <*errors.errorString | 0xc001810590>: { s: "pod \"pod-92fb967d-00bb-4449-b567-f10d76ce462f\" is not Running: timed out waiting for the condition", } pod "pod-92fb967d-00bb-4449-b567-f10d76ce462f" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:727 ------------------------------ {"msg":"FAILED [sig-storage] PersistentVolumes-local [Volume type: tmpfs] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2","total":-1,"completed":2,"skipped":198,"failed":3,"failures":["[sig-storage] PersistentVolumes-local [Volume type: blockfswithoutformat] One pod requesting one prebound PVC should be able to mount volume and read from pod1","[sig-storage] PersistentVolumes-local [Volume type: blockfswithformat] One pod requesting one prebound PVC should be able to mount volume and read from pod1","[sig-storage] PersistentVolumes-local [Volume type: tmpfs] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2"]} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:50:15.612: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename csi-mock-volumes STEP: Waiting for a default service account to be provisioned in namespace [It] should expand volume by restarting pod if attach=on, nodeExpansion=on /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:455 STEP: Building a driver namespace object, basename csi-mock-volumes-253 STEP: Waiting for a default service account to be provisioned in namespace STEP: deploying csi mock driver May 15 01:50:15.680: INFO: creating *v1.ServiceAccount: csi-mock-volumes-253-4213/csi-attacher May 15 01:50:15.683: INFO: creating *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-253 May 15 01:50:15.683: INFO: Define cluster role external-attacher-runner-csi-mock-volumes-253 May 15 01:50:15.685: INFO: creating *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-253 May 15 01:50:15.688: INFO: creating *v1.Role: csi-mock-volumes-253-4213/external-attacher-cfg-csi-mock-volumes-253 May 15 01:50:15.690: INFO: creating *v1.RoleBinding: csi-mock-volumes-253-4213/csi-attacher-role-cfg May 15 01:50:15.692: INFO: creating *v1.ServiceAccount: csi-mock-volumes-253-4213/csi-provisioner May 15 01:50:15.696: INFO: creating *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-253 May 15 01:50:15.696: INFO: Define cluster role external-provisioner-runner-csi-mock-volumes-253 May 15 01:50:15.699: INFO: creating *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-253 May 15 01:50:15.702: INFO: creating *v1.Role: csi-mock-volumes-253-4213/external-provisioner-cfg-csi-mock-volumes-253 May 15 01:50:15.704: INFO: creating *v1.RoleBinding: csi-mock-volumes-253-4213/csi-provisioner-role-cfg May 15 01:50:15.707: INFO: creating *v1.ServiceAccount: csi-mock-volumes-253-4213/csi-resizer May 15 01:50:15.709: INFO: creating *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-253 May 15 01:50:15.709: INFO: Define cluster role external-resizer-runner-csi-mock-volumes-253 May 15 01:50:15.712: INFO: creating *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-253 May 15 01:50:15.715: INFO: creating *v1.Role: csi-mock-volumes-253-4213/external-resizer-cfg-csi-mock-volumes-253 May 15 01:50:15.718: INFO: creating *v1.RoleBinding: csi-mock-volumes-253-4213/csi-resizer-role-cfg May 15 01:50:15.720: INFO: creating *v1.ServiceAccount: csi-mock-volumes-253-4213/csi-mock May 15 01:50:15.723: INFO: creating *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-253 May 15 01:50:15.725: INFO: creating *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-253 May 15 01:50:15.727: INFO: creating *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-253 May 15 01:50:15.729: INFO: creating *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-253 May 15 01:50:15.731: INFO: creating *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-253 May 15 01:50:15.734: INFO: creating *v1.StorageClass: csi-mock-sc-csi-mock-volumes-253 May 15 01:50:15.736: INFO: creating *v1.StatefulSet: csi-mock-volumes-253-4213/csi-mockplugin May 15 01:50:15.740: INFO: creating *v1.StatefulSet: csi-mock-volumes-253-4213/csi-mockplugin-attacher May 15 01:50:15.743: INFO: creating *v1.StatefulSet: csi-mock-volumes-253-4213/csi-mockplugin-resizer May 15 01:50:15.746: INFO: waiting for CSIDriver csi-mock-csi-mock-volumes-253 to register on node node1 STEP: Creating pod May 15 01:50:25.263: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil May 15 01:50:25.269: INFO: Waiting up to 5m0s for PersistentVolumeClaims [pvc-2zxlh] to have phase Bound May 15 01:50:25.271: INFO: PersistentVolumeClaim pvc-2zxlh found but phase is Pending instead of Bound. May 15 01:50:27.274: INFO: PersistentVolumeClaim pvc-2zxlh found and phase=Bound (2.005510769s) STEP: Expanding current pvc STEP: Waiting for persistent volume resize to finish STEP: Checking for conditions on pvc STEP: Deleting the previously created pod May 15 01:50:49.307: INFO: Deleting pod "pvc-volume-tester-bd8nn" in namespace "csi-mock-volumes-253" May 15 01:50:49.311: INFO: Wait up to 5m0s for pod "pvc-volume-tester-bd8nn" to be fully deleted STEP: Creating a new pod with same volume STEP: Waiting for PVC resize to finish STEP: Deleting pod pvc-volume-tester-bd8nn May 15 01:50:57.333: INFO: Deleting pod "pvc-volume-tester-bd8nn" in namespace "csi-mock-volumes-253" STEP: Deleting pod pvc-volume-tester-5nsnk May 15 01:50:57.335: INFO: Deleting pod "pvc-volume-tester-5nsnk" in namespace "csi-mock-volumes-253" May 15 01:50:57.339: INFO: Wait up to 5m0s for pod "pvc-volume-tester-5nsnk" to be fully deleted STEP: Deleting claim pvc-2zxlh May 15 01:51:11.353: INFO: Waiting up to 2m0s for PersistentVolume pvc-7c5791fe-3ce6-4834-a077-b3276ec8a615 to get deleted May 15 01:51:11.355: INFO: PersistentVolume pvc-7c5791fe-3ce6-4834-a077-b3276ec8a615 found and phase=Bound (1.914642ms) May 15 01:51:13.359: INFO: PersistentVolume pvc-7c5791fe-3ce6-4834-a077-b3276ec8a615 was removed STEP: Deleting storageclass csi-mock-volumes-253-sc STEP: Cleaning up resources STEP: deleting the test namespace: csi-mock-volumes-253 STEP: Waiting for namespaces [csi-mock-volumes-253] to vanish STEP: uninstalling csi mock driver May 15 01:51:19.375: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-253-4213/csi-attacher May 15 01:51:19.379: INFO: deleting *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-253 May 15 01:51:19.383: INFO: deleting *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-253 May 15 01:51:19.387: INFO: deleting *v1.Role: csi-mock-volumes-253-4213/external-attacher-cfg-csi-mock-volumes-253 May 15 01:51:19.391: INFO: deleting *v1.RoleBinding: csi-mock-volumes-253-4213/csi-attacher-role-cfg May 15 01:51:19.396: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-253-4213/csi-provisioner May 15 01:51:19.400: INFO: deleting *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-253 May 15 01:51:19.404: INFO: deleting *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-253 May 15 01:51:19.409: INFO: deleting *v1.Role: csi-mock-volumes-253-4213/external-provisioner-cfg-csi-mock-volumes-253 May 15 01:51:19.414: INFO: deleting *v1.RoleBinding: csi-mock-volumes-253-4213/csi-provisioner-role-cfg May 15 01:51:19.418: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-253-4213/csi-resizer May 15 01:51:19.424: INFO: deleting *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-253 May 15 01:51:19.428: INFO: deleting *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-253 May 15 01:51:19.432: INFO: deleting *v1.Role: csi-mock-volumes-253-4213/external-resizer-cfg-csi-mock-volumes-253 May 15 01:51:19.436: INFO: deleting *v1.RoleBinding: csi-mock-volumes-253-4213/csi-resizer-role-cfg May 15 01:51:19.440: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-253-4213/csi-mock May 15 01:51:19.444: INFO: deleting *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-253 May 15 01:51:19.448: INFO: deleting *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-253 May 15 01:51:19.452: INFO: deleting *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-253 May 15 01:51:19.456: INFO: deleting *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-253 May 15 01:51:19.459: INFO: deleting *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-253 May 15 01:51:19.464: INFO: deleting *v1.StorageClass: csi-mock-sc-csi-mock-volumes-253 May 15 01:51:19.468: INFO: deleting *v1.StatefulSet: csi-mock-volumes-253-4213/csi-mockplugin May 15 01:51:19.473: INFO: deleting *v1.StatefulSet: csi-mock-volumes-253-4213/csi-mockplugin-attacher May 15 01:51:19.477: INFO: deleting *v1.StatefulSet: csi-mock-volumes-253-4213/csi-mockplugin-resizer STEP: deleting the driver namespace: csi-mock-volumes-253-4213 STEP: Waiting for namespaces [csi-mock-volumes-253-4213] to vanish [AfterEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:51:35.491: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "csi-mock-volumes-253" for this suite. May 15 01:51:35.497: INFO: Namespace csi-mock-volumes-253 was already deleted STEP: Destroying namespace "csi-mock-volumes-253-4213" for this suite. May 15 01:51:35.499: INFO: Namespace csi-mock-volumes-253-4213 was already deleted • [SLOW TEST:79.887 seconds] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 CSI Volume expansion /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:426 should expand volume by restarting pod if attach=on, nodeExpansion=on /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:455 ------------------------------ {"msg":"PASSED [sig-storage] CSI mock volume CSI Volume expansion should expand volume by restarting pod if attach=on, nodeExpansion=on","total":-1,"completed":6,"skipped":704,"failed":2,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] One pod requesting one prebound PVC should be able to mount volume and read from pod1","[sig-storage] PersistentVolumes-local [Volume type: block] One pod requesting one prebound PVC should be able to mount volume and write from pod1"]} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] EmptyDir volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:51:35.564: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename emptydir STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] when FSGroup is specified [LinuxOnly] [NodeFeature:FSGroup] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/empty_dir.go:47 [It] volume on tmpfs should have the correct mode using FSGroup /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/empty_dir.go:72 STEP: Creating a pod to test emptydir volume type on tmpfs May 15 01:51:35.601: INFO: Waiting up to 5m0s for pod "pod-e8d1d129-9f02-4ab9-bc67-62bf1b562adb" in namespace "emptydir-2702" to be "Succeeded or Failed" May 15 01:51:35.605: INFO: Pod "pod-e8d1d129-9f02-4ab9-bc67-62bf1b562adb": Phase="Pending", Reason="", readiness=false. Elapsed: 3.796225ms May 15 01:51:37.609: INFO: Pod "pod-e8d1d129-9f02-4ab9-bc67-62bf1b562adb": Phase="Pending", Reason="", readiness=false. Elapsed: 2.007687791s May 15 01:51:39.612: INFO: Pod "pod-e8d1d129-9f02-4ab9-bc67-62bf1b562adb": Phase="Succeeded", Reason="", readiness=false. Elapsed: 4.010902261s STEP: Saw pod success May 15 01:51:39.612: INFO: Pod "pod-e8d1d129-9f02-4ab9-bc67-62bf1b562adb" satisfied condition "Succeeded or Failed" May 15 01:51:39.615: INFO: Trying to get logs from node node2 pod pod-e8d1d129-9f02-4ab9-bc67-62bf1b562adb container test-container: STEP: delete the pod May 15 01:51:39.629: INFO: Waiting for pod pod-e8d1d129-9f02-4ab9-bc67-62bf1b562adb to disappear May 15 01:51:39.631: INFO: Pod pod-e8d1d129-9f02-4ab9-bc67-62bf1b562adb no longer exists [AfterEach] [sig-storage] EmptyDir volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:51:39.631: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "emptydir-2702" for this suite. • ------------------------------ {"msg":"PASSED [sig-storage] EmptyDir volumes when FSGroup is specified [LinuxOnly] [NodeFeature:FSGroup] volume on tmpfs should have the correct mode using FSGroup","total":-1,"completed":7,"skipped":735,"failed":2,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] One pod requesting one prebound PVC should be able to mount volume and read from pod1","[sig-storage] PersistentVolumes-local [Volume type: block] One pod requesting one prebound PVC should be able to mount volume and write from pod1"]} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:51:29.836: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: dir-bindmounted] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes May 15 01:51:33.887: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir /tmp/local-volume-test-eb3ca12c-053b-4894-ab9c-240a44a2d1df && mount --bind /tmp/local-volume-test-eb3ca12c-053b-4894-ab9c-240a44a2d1df /tmp/local-volume-test-eb3ca12c-053b-4894-ab9c-240a44a2d1df] Namespace:persistent-local-volumes-test-136 PodName:hostexec-node1-jh7mw ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:51:33.887: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:51:34.009: INFO: Creating a PV followed by a PVC May 15 01:51:34.017: INFO: Waiting for PV local-pvt9q8g to bind to PVC pvc-xlzp5 May 15 01:51:34.017: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-xlzp5] to have phase Bound May 15 01:51:34.019: INFO: PersistentVolumeClaim pvc-xlzp5 found but phase is Pending instead of Bound. May 15 01:51:36.023: INFO: PersistentVolumeClaim pvc-xlzp5 found but phase is Pending instead of Bound. May 15 01:51:38.026: INFO: PersistentVolumeClaim pvc-xlzp5 found but phase is Pending instead of Bound. May 15 01:51:40.029: INFO: PersistentVolumeClaim pvc-xlzp5 found but phase is Pending instead of Bound. May 15 01:51:42.033: INFO: PersistentVolumeClaim pvc-xlzp5 found but phase is Pending instead of Bound. May 15 01:51:44.037: INFO: PersistentVolumeClaim pvc-xlzp5 found and phase=Bound (10.019615121s) May 15 01:51:44.037: INFO: Waiting up to 3m0s for PersistentVolume local-pvt9q8g to have phase Bound May 15 01:51:44.039: INFO: PersistentVolume local-pvt9q8g found and phase=Bound (2.660108ms) [BeforeEach] Set fsGroup for local volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:257 [It] should set different fsGroup for second pod if first pod is deleted /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:282 May 15 01:51:44.045: INFO: Disabled temporarily, reopen after #73168 is fixed [AfterEach] [Volume type: dir-bindmounted] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:51:44.046: INFO: Deleting PersistentVolumeClaim "pvc-xlzp5" May 15 01:51:44.052: INFO: Deleting PersistentVolume "local-pvt9q8g" STEP: Removing the test directory May 15 01:51:44.056: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c umount /tmp/local-volume-test-eb3ca12c-053b-4894-ab9c-240a44a2d1df && rm -r /tmp/local-volume-test-eb3ca12c-053b-4894-ab9c-240a44a2d1df] Namespace:persistent-local-volumes-test-136 PodName:hostexec-node1-jh7mw ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:51:44.056: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:51:44.289: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-136" for this suite. S [SKIPPING] [14.461 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: dir-bindmounted] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 Set fsGroup for local volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:256 should set different fsGroup for second pod if first pod is deleted [It] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:282 Disabled temporarily, reopen after #73168 is fixed /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:283 ------------------------------ SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:49:32.765: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: blockfswithformat] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes STEP: Creating block device on node "node1" using path "/tmp/local-volume-test-8811626d-1178-4164-b2d6-2d97dbe014e7" May 15 01:49:38.820: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-8811626d-1178-4164-b2d6-2d97dbe014e7 && dd if=/dev/zero of=/tmp/local-volume-test-8811626d-1178-4164-b2d6-2d97dbe014e7/file bs=4096 count=5120 && losetup -f /tmp/local-volume-test-8811626d-1178-4164-b2d6-2d97dbe014e7/file] Namespace:persistent-local-volumes-test-9759 PodName:hostexec-node1-8lfk7 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:49:38.820: INFO: >>> kubeConfig: /root/.kube/config May 15 01:49:38.962: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c E2E_LOOP_DEV=$(losetup | grep /tmp/local-volume-test-8811626d-1178-4164-b2d6-2d97dbe014e7/file | awk '{ print $1 }') 2>&1 > /dev/null && echo ${E2E_LOOP_DEV}] Namespace:persistent-local-volumes-test-9759 PodName:hostexec-node1-8lfk7 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:49:38.962: INFO: >>> kubeConfig: /root/.kube/config May 15 01:49:39.086: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkfs -t ext4 /dev/loop1 && mount -t ext4 /dev/loop1 /tmp/local-volume-test-8811626d-1178-4164-b2d6-2d97dbe014e7 && chmod o+rwx /tmp/local-volume-test-8811626d-1178-4164-b2d6-2d97dbe014e7] Namespace:persistent-local-volumes-test-9759 PodName:hostexec-node1-8lfk7 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:49:39.086: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:49:39.371: INFO: Creating a PV followed by a PVC May 15 01:49:39.377: INFO: Waiting for PV local-pvwnnhf to bind to PVC pvc-2gm25 May 15 01:49:39.377: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-2gm25] to have phase Bound May 15 01:49:39.380: INFO: PersistentVolumeClaim pvc-2gm25 found but phase is Pending instead of Bound. May 15 01:49:41.383: INFO: PersistentVolumeClaim pvc-2gm25 found but phase is Pending instead of Bound. May 15 01:49:43.386: INFO: PersistentVolumeClaim pvc-2gm25 found and phase=Bound (4.00887895s) May 15 01:49:43.386: INFO: Waiting up to 3m0s for PersistentVolume local-pvwnnhf to have phase Bound May 15 01:49:43.389: INFO: PersistentVolume local-pvwnnhf found and phase=Bound (2.207925ms) [BeforeEach] One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:211 STEP: Creating pod1 STEP: Creating a pod May 15 01:51:43.416: FAIL: Unexpected error: <*errors.errorString | 0xc002226e60>: { s: "pod \"pod-e5bfb405-29b8-435c-ab5f-54edab17f88c\" is not Running: timed out waiting for the condition", } pod "pod-e5bfb405-29b8-435c-ab5f-54edab17f88c" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage.glob..func20.2.3.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 +0x119 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc00362b080) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc00362b080) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc00362b080, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:223 STEP: Deleting pod1 STEP: Deleting pod pod-e5bfb405-29b8-435c-ab5f-54edab17f88c in namespace persistent-local-volumes-test-9759 [AfterEach] [Volume type: blockfswithformat] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:51:43.423: INFO: Deleting PersistentVolumeClaim "pvc-2gm25" May 15 01:51:43.428: INFO: Deleting PersistentVolume "local-pvwnnhf" May 15 01:51:43.432: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c umount /tmp/local-volume-test-8811626d-1178-4164-b2d6-2d97dbe014e7] Namespace:persistent-local-volumes-test-9759 PodName:hostexec-node1-8lfk7 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:51:43.432: INFO: >>> kubeConfig: /root/.kube/config May 15 01:51:43.599: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c E2E_LOOP_DEV=$(losetup | grep /tmp/local-volume-test-8811626d-1178-4164-b2d6-2d97dbe014e7/file | awk '{ print $1 }') 2>&1 > /dev/null && echo ${E2E_LOOP_DEV}] Namespace:persistent-local-volumes-test-9759 PodName:hostexec-node1-8lfk7 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:51:43.599: INFO: >>> kubeConfig: /root/.kube/config STEP: Tear down block device "/dev/loop1" on node "node1" at path /tmp/local-volume-test-8811626d-1178-4164-b2d6-2d97dbe014e7/file May 15 01:51:43.726: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c losetup -d /dev/loop1] Namespace:persistent-local-volumes-test-9759 PodName:hostexec-node1-8lfk7 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:51:43.726: INFO: >>> kubeConfig: /root/.kube/config STEP: Removing the test directory /tmp/local-volume-test-8811626d-1178-4164-b2d6-2d97dbe014e7 May 15 01:51:43.994: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-8811626d-1178-4164-b2d6-2d97dbe014e7] Namespace:persistent-local-volumes-test-9759 PodName:hostexec-node1-8lfk7 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:51:43.994: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "persistent-local-volumes-test-9759". STEP: Found 13 events. May 15 01:51:44.157: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for hostexec-node1-8lfk7: { } Scheduled: Successfully assigned persistent-local-volumes-test-9759/hostexec-node1-8lfk7 to node1 May 15 01:51:44.157: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-e5bfb405-29b8-435c-ab5f-54edab17f88c: { } Scheduled: Successfully assigned persistent-local-volumes-test-9759/pod-e5bfb405-29b8-435c-ab5f-54edab17f88c to node1 May 15 01:51:44.157: INFO: At 2021-05-15 01:49:34 +0000 UTC - event for hostexec-node1-8lfk7: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 15 01:51:44.157: INFO: At 2021-05-15 01:49:36 +0000 UTC - event for hostexec-node1-8lfk7: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 1.207959819s May 15 01:51:44.157: INFO: At 2021-05-15 01:49:36 +0000 UTC - event for hostexec-node1-8lfk7: {kubelet node1} Created: Created container agnhost-container May 15 01:51:44.157: INFO: At 2021-05-15 01:49:36 +0000 UTC - event for hostexec-node1-8lfk7: {kubelet node1} Started: Started container agnhost-container May 15 01:51:44.157: INFO: At 2021-05-15 01:49:39 +0000 UTC - event for pvc-2gm25: {persistentvolume-controller } ProvisioningFailed: no volume plugin matched name: kubernetes.io/no-provisioner May 15 01:51:44.157: INFO: At 2021-05-15 01:49:45 +0000 UTC - event for pod-e5bfb405-29b8-435c-ab5f-54edab17f88c: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:51:44.157: INFO: At 2021-05-15 01:49:45 +0000 UTC - event for pod-e5bfb405-29b8-435c-ab5f-54edab17f88c: {multus } AddedInterface: Add eth0 [10.244.3.171/24] May 15 01:51:44.157: INFO: At 2021-05-15 01:49:46 +0000 UTC - event for pod-e5bfb405-29b8-435c-ab5f-54edab17f88c: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:51:44.157: INFO: At 2021-05-15 01:49:46 +0000 UTC - event for pod-e5bfb405-29b8-435c-ab5f-54edab17f88c: {kubelet node1} Failed: Error: ErrImagePull May 15 01:51:44.157: INFO: At 2021-05-15 01:49:46 +0000 UTC - event for pod-e5bfb405-29b8-435c-ab5f-54edab17f88c: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 15 01:51:44.157: INFO: At 2021-05-15 01:49:46 +0000 UTC - event for pod-e5bfb405-29b8-435c-ab5f-54edab17f88c: {kubelet node1} Failed: Error: ImagePullBackOff May 15 01:51:44.159: INFO: POD NODE PHASE GRACE CONDITIONS May 15 01:51:44.159: INFO: hostexec-node1-8lfk7 node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:49:32 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:49:37 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:49:37 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:49:32 +0000 UTC }] May 15 01:51:44.159: INFO: pod-e5bfb405-29b8-435c-ab5f-54edab17f88c node1 Pending 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:49:43 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:49:43 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:49:43 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:49:43 +0000 UTC }] May 15 01:51:44.159: INFO: May 15 01:51:44.164: INFO: Logging node info for node master1 May 15 01:51:44.166: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 159385 0 2021-05-14 19:56:35 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:38 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:38 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:38 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:51:38 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[@ :],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:51:44.167: INFO: Logging kubelet events for node master1 May 15 01:51:44.171: INFO: Logging pods the kubelet thinks is on node master1 May 15 01:51:44.180: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.180: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:51:44.180: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:51:44.180: INFO: Init container install-cni ready: true, restart count 0 May 15 01:51:44.180: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:51:44.180: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:51:44.180: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:51:44.180: INFO: Container node-exporter ready: true, restart count 0 May 15 01:51:44.180: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.180: INFO: Container kube-scheduler ready: true, restart count 0 May 15 01:51:44.180: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 15 01:51:44.180: INFO: Container docker-registry ready: true, restart count 0 May 15 01:51:44.180: INFO: Container nginx ready: true, restart count 0 May 15 01:51:44.180: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.180: INFO: Container nfd-controller ready: true, restart count 0 May 15 01:51:44.180: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.180: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:51:44.180: INFO: kubernetes-metrics-scraper-678c97765c-fswrn started at 2021-05-15 00:18:49 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.180: INFO: Container kubernetes-metrics-scraper ready: true, restart count 0 May 15 01:51:44.180: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.180: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:51:44.180: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.180: INFO: Container kube-multus ready: true, restart count 1 W0515 01:51:44.193943 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:51:44.220: INFO: Latency metrics for node master1 May 15 01:51:44.220: INFO: Logging node info for node master2 May 15 01:51:44.223: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 159366 0 2021-05-14 19:57:04 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:36 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:36 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:36 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:51:36 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:51:44.224: INFO: Logging kubelet events for node master2 May 15 01:51:44.228: INFO: Logging pods the kubelet thinks is on node master2 May 15 01:51:44.239: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.239: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:51:44.239: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.239: INFO: Container kube-multus ready: true, restart count 1 May 15 01:51:44.239: INFO: coredns-7677f9bb54-96w24 started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.239: INFO: Container coredns ready: true, restart count 0 May 15 01:51:44.239: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:51:44.239: INFO: Init container install-cni ready: true, restart count 0 May 15 01:51:44.239: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:51:44.239: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.239: INFO: Container autoscaler ready: true, restart count 2 May 15 01:51:44.239: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:51:44.239: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:51:44.239: INFO: Container node-exporter ready: true, restart count 0 May 15 01:51:44.239: INFO: prometheus-operator-5bb8cb9d8f-fqb87 started at 2021-05-15 00:18:49 +0000 UTC (0+2 container statuses recorded) May 15 01:51:44.239: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:51:44.239: INFO: Container prometheus-operator ready: true, restart count 0 May 15 01:51:44.239: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.239: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:51:44.239: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.239: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:51:44.239: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.239: INFO: Container kube-scheduler ready: true, restart count 2 W0515 01:51:44.252521 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:51:44.277: INFO: Latency metrics for node master2 May 15 01:51:44.277: INFO: Logging node info for node master3 May 15 01:51:44.280: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 159339 0 2021-05-14 19:57:14 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234767872 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324603904 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:34 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:34 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:34 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:51:34 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:51:44.280: INFO: Logging kubelet events for node master3 May 15 01:51:44.285: INFO: Logging pods the kubelet thinks is on node master3 May 15 01:51:44.292: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:51:44.292: INFO: Init container install-cni ready: true, restart count 0 May 15 01:51:44.292: INFO: Container kube-flannel ready: true, restart count 2 May 15 01:51:44.292: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.292: INFO: Container kube-multus ready: true, restart count 1 May 15 01:51:44.292: INFO: coredns-7677f9bb54-rpj8c started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.292: INFO: Container coredns ready: true, restart count 0 May 15 01:51:44.292: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:51:44.292: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:51:44.292: INFO: Container node-exporter ready: true, restart count 0 May 15 01:51:44.292: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.292: INFO: Container kube-controller-manager ready: true, restart count 3 May 15 01:51:44.292: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.292: INFO: Container kube-scheduler ready: true, restart count 3 May 15 01:51:44.292: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.293: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:51:44.293: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.293: INFO: Container kube-proxy ready: true, restart count 1 W0515 01:51:44.305104 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:51:44.333: INFO: Latency metrics for node master3 May 15 01:51:44.333: INFO: Logging node info for node node1 May 15 01:51:44.337: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 159365 0 2021-05-14 19:58:21 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1602":"csi-mock-csi-mock-volumes-1602","csi-mock-csi-mock-volumes-253":"csi-mock-csi-mock-volumes-253","csi-mock-csi-mock-volumes-7734":"csi-mock-csi-mock-volumes-7734","csi-mock-csi-mock-volumes-895":"csi-mock-csi-mock-volumes-895","csi-mock-csi-mock-volumes-9728":"csi-mock-csi-mock-volumes-9728","csi-mock-csi-mock-volumes-9809":"csi-mock-csi-mock-volumes-9809"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:00 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:50:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:51:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:36 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:36 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:36 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:51:36 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[@ :],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:51:44.338: INFO: Logging kubelet events for node node1 May 15 01:51:44.342: INFO: Logging pods the kubelet thinks is on node node1 May 15 01:51:44.370: INFO: pod-11760848-bbb5-4ae1-af82-a7a2a4af6a9c started at 2021-05-15 01:50:59 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.370: INFO: Container write-pod ready: false, restart count 0 May 15 01:51:44.370: INFO: pod-e5bfb405-29b8-435c-ab5f-54edab17f88c started at 2021-05-15 01:49:43 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.370: INFO: Container write-pod ready: false, restart count 0 May 15 01:51:44.370: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:51:44.370: INFO: Init container install-cni ready: true, restart count 0 May 15 01:51:44.370: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:51:44.370: INFO: node-feature-discovery-worker-bw8zg started at 2021-05-15 00:18:56 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.370: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:51:44.370: INFO: collectd-mrzps started at 2021-05-15 00:19:22 +0000 UTC (0+3 container statuses recorded) May 15 01:51:44.370: INFO: Container collectd ready: true, restart count 0 May 15 01:51:44.370: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:51:44.370: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:51:44.370: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.370: INFO: Container kube-multus ready: true, restart count 1 May 15 01:51:44.370: INFO: prometheus-k8s-0 started at 2021-05-15 00:19:01 +0000 UTC (0+5 container statuses recorded) May 15 01:51:44.370: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 15 01:51:44.370: INFO: Container grafana ready: true, restart count 0 May 15 01:51:44.370: INFO: Container prometheus ready: true, restart count 26 May 15 01:51:44.370: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 15 01:51:44.370: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 15 01:51:44.370: INFO: pod-6fec197a-289d-4046-b7a7-00d385f2dc10 started at 2021-05-15 01:50:35 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.370: INFO: Container write-pod ready: false, restart count 0 May 15 01:51:44.370: INFO: hostexec-node1-ktptr started at 2021-05-15 01:50:37 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.370: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:51:44.370: INFO: csi-mockplugin-0 started at 2021-05-15 01:51:15 +0000 UTC (0+3 container statuses recorded) May 15 01:51:44.370: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:51:44.370: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:51:44.370: INFO: Container mock ready: true, restart count 0 May 15 01:51:44.370: INFO: node-exporter-flvqz started at 2021-05-15 00:18:55 +0000 UTC (0+2 container statuses recorded) May 15 01:51:44.370: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:51:44.370: INFO: Container node-exporter ready: true, restart count 0 May 15 01:51:44.370: INFO: pod-58f129b2-416d-48d8-93b6-2628e4cc65d5 started at 2021-05-15 01:50:35 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.370: INFO: Container write-pod ready: false, restart count 0 May 15 01:51:44.370: INFO: pod-e94cee9f-0eca-47df-9ce8-1d446f676d35 started at 2021-05-15 01:49:59 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.370: INFO: Container write-pod ready: false, restart count 0 May 15 01:51:44.370: INFO: pod-9566f4e9-ff2d-489c-94bf-709e28a35a7d started at 2021-05-15 01:50:43 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.370: INFO: Container write-pod ready: false, restart count 0 May 15 01:51:44.370: INFO: cmk-4s6dm started at 2021-05-15 00:18:54 +0000 UTC (0+2 container statuses recorded) May 15 01:51:44.370: INFO: Container nodereport ready: true, restart count 0 May 15 01:51:44.370: INFO: Container reconcile ready: true, restart count 0 May 15 01:51:44.370: INFO: hostexec-node1-g9jvw started at 2021-05-15 01:49:51 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.370: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:51:44.370: INFO: hostexec-node1-8lfk7 started at 2021-05-15 01:49:32 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.370: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:51:44.370: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.370: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:51:44.370: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4vxcc started at 2021-05-15 00:19:00 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.370: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:51:44.370: INFO: hostexec-node1-vlzvd started at 2021-05-15 01:51:39 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.370: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:51:44.370: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.370: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:51:44.370: INFO: hostexec-node1-jh7mw started at 2021-05-15 01:51:29 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.370: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:51:44.370: INFO: hostexec-node1-2l9b5 started at 2021-05-15 01:50:29 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.370: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:51:44.370: INFO: hostexec-node1-dwvtr started at 2021-05-15 01:50:31 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.371: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:51:44.371: INFO: hostexec-node1-zbxmq started at 2021-05-15 01:50:51 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.371: INFO: Container agnhost-container ready: true, restart count 0 W0515 01:51:44.384422 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:51:44.425: INFO: Latency metrics for node node1 May 15 01:51:44.425: INFO: Logging node info for node node2 May 15 01:51:44.428: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 159413 0 2021-05-14 19:58:22 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1663":"csi-mock-csi-mock-volumes-1663","csi-mock-csi-mock-volumes-3052":"csi-mock-csi-mock-volumes-3052","csi-mock-csi-mock-volumes-5678":"csi-mock-csi-mock-volumes-5678","csi-mock-csi-mock-volumes-8760":"csi-mock-csi-mock-volumes-8760","csi-mock-csi-mock-volumes-9624":"csi-mock-csi-mock-volumes-9624"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:46 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:44:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:46:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:io.kubernetes.storage.mock/node":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:40 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:40 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:40 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:51:40 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:51:44.429: INFO: Logging kubelet events for node node2 May 15 01:51:44.433: INFO: Logging pods the kubelet thinks is on node node2 May 15 01:51:44.444: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.444: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:51:44.444: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:51:44.444: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:51:44.444: INFO: Container node-exporter ready: true, restart count 0 May 15 01:51:44.444: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:51:44.444: INFO: Init container install-cni ready: true, restart count 1 May 15 01:51:44.444: INFO: Container kube-flannel ready: true, restart count 4 May 15 01:51:44.444: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 15 01:51:44.444: INFO: Container collectd ready: true, restart count 0 May 15 01:51:44.444: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:51:44.444: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:51:44.444: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 15 01:51:44.444: INFO: Container tas-controller ready: true, restart count 0 May 15 01:51:44.444: INFO: Container tas-extender ready: true, restart count 0 May 15 01:51:44.444: INFO: pod-subpath-test-configmap-lp9m started at 2021-05-15 01:50:05 +0000 UTC (1+2 container statuses recorded) May 15 01:51:44.444: INFO: Init container init-volume-configmap-lp9m ready: false, restart count 0 May 15 01:51:44.444: INFO: Container test-container-subpath-configmap-lp9m ready: false, restart count 0 May 15 01:51:44.444: INFO: Container test-container-volume-configmap-lp9m ready: false, restart count 0 May 15 01:51:44.444: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.444: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:51:44.444: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 15 01:51:44.444: INFO: Container nodereport ready: true, restart count 0 May 15 01:51:44.444: INFO: Container reconcile ready: true, restart count 0 May 15 01:51:44.444: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.444: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:51:44.444: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.444: INFO: Container cmk-webhook ready: true, restart count 0 May 15 01:51:44.444: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 15 01:51:44.444: INFO: Container discover ready: false, restart count 0 May 15 01:51:44.444: INFO: Container init ready: false, restart count 0 May 15 01:51:44.444: INFO: Container install ready: false, restart count 0 May 15 01:51:44.444: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.444: INFO: Container kube-multus ready: true, restart count 1 May 15 01:51:44.444: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.444: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 15 01:51:44.444: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:51:44.444: INFO: Container kube-proxy ready: true, restart count 2 W0515 01:51:44.458049 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:51:44.513: INFO: Latency metrics for node node2 May 15 01:51:44.513: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-9759" for this suite. • Failure in Spec Setup (BeforeEach) [131.757 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: blockfswithformat] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:205 should be able to mount volume and write from pod1 [BeforeEach] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:234 May 15 01:51:43.416: Unexpected error: <*errors.errorString | 0xc002226e60>: { s: "pod \"pod-e5bfb405-29b8-435c-ab5f-54edab17f88c\" is not Running: timed out waiting for the condition", } pod "pod-e5bfb405-29b8-435c-ab5f-54edab17f88c" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 ------------------------------ {"msg":"FAILED [sig-storage] PersistentVolumes-local [Volume type: blockfswithformat] One pod requesting one prebound PVC should be able to mount volume and write from pod1","total":-1,"completed":3,"skipped":436,"failed":2,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] One pod requesting one prebound PVC should be able to mount volume and write from pod1","[sig-storage] PersistentVolumes-local [Volume type: blockfswithformat] One pod requesting one prebound PVC should be able to mount volume and write from pod1"]} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:51:44.755: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: dir-link] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes May 15 01:51:48.800: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir /tmp/local-volume-test-3b1815c0-28ce-4e76-8149-45ab050250bf-backend && ln -s /tmp/local-volume-test-3b1815c0-28ce-4e76-8149-45ab050250bf-backend /tmp/local-volume-test-3b1815c0-28ce-4e76-8149-45ab050250bf] Namespace:persistent-local-volumes-test-1925 PodName:hostexec-node1-5sh66 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:51:48.800: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:51:48.947: INFO: Creating a PV followed by a PVC May 15 01:51:48.953: INFO: Waiting for PV local-pv98mwc to bind to PVC pvc-m8gxm May 15 01:51:48.953: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-m8gxm] to have phase Bound May 15 01:51:48.955: INFO: PersistentVolumeClaim pvc-m8gxm found but phase is Pending instead of Bound. May 15 01:51:50.959: INFO: PersistentVolumeClaim pvc-m8gxm found but phase is Pending instead of Bound. May 15 01:51:52.962: INFO: PersistentVolumeClaim pvc-m8gxm found but phase is Pending instead of Bound. May 15 01:51:54.965: INFO: PersistentVolumeClaim pvc-m8gxm found but phase is Pending instead of Bound. May 15 01:51:56.969: INFO: PersistentVolumeClaim pvc-m8gxm found but phase is Pending instead of Bound. May 15 01:51:58.972: INFO: PersistentVolumeClaim pvc-m8gxm found and phase=Bound (10.018922807s) May 15 01:51:58.972: INFO: Waiting up to 3m0s for PersistentVolume local-pv98mwc to have phase Bound May 15 01:51:58.974: INFO: PersistentVolume local-pv98mwc found and phase=Bound (2.135893ms) [BeforeEach] Set fsGroup for local volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:257 [It] should set different fsGroup for second pod if first pod is deleted /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:282 May 15 01:51:58.978: INFO: Disabled temporarily, reopen after #73168 is fixed [AfterEach] [Volume type: dir-link] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:51:58.980: INFO: Deleting PersistentVolumeClaim "pvc-m8gxm" May 15 01:51:58.985: INFO: Deleting PersistentVolume "local-pv98mwc" STEP: Removing the test directory May 15 01:51:58.989: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-3b1815c0-28ce-4e76-8149-45ab050250bf && rm -r /tmp/local-volume-test-3b1815c0-28ce-4e76-8149-45ab050250bf-backend] Namespace:persistent-local-volumes-test-1925 PodName:hostexec-node1-5sh66 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:51:58.989: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:51:59.114: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-1925" for this suite. S [SKIPPING] [14.369 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: dir-link] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 Set fsGroup for local volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:256 should set different fsGroup for second pod if first pod is deleted [It] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:282 Disabled temporarily, reopen after #73168 is fixed /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:283 ------------------------------ SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:51:44.459: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: blockfswithoutformat] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes STEP: Creating block device on node "node1" using path "/tmp/local-volume-test-1b2c98a3-07c1-4c69-9160-dbada5bb7a62" May 15 01:51:48.513: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-1b2c98a3-07c1-4c69-9160-dbada5bb7a62 && dd if=/dev/zero of=/tmp/local-volume-test-1b2c98a3-07c1-4c69-9160-dbada5bb7a62/file bs=4096 count=5120 && losetup -f /tmp/local-volume-test-1b2c98a3-07c1-4c69-9160-dbada5bb7a62/file] Namespace:persistent-local-volumes-test-5616 PodName:hostexec-node1-kstrk ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:51:48.513: INFO: >>> kubeConfig: /root/.kube/config May 15 01:51:48.767: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c E2E_LOOP_DEV=$(losetup | grep /tmp/local-volume-test-1b2c98a3-07c1-4c69-9160-dbada5bb7a62/file | awk '{ print $1 }') 2>&1 > /dev/null && echo ${E2E_LOOP_DEV}] Namespace:persistent-local-volumes-test-5616 PodName:hostexec-node1-kstrk ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:51:48.767: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:51:48.887: INFO: Creating a PV followed by a PVC May 15 01:51:48.894: INFO: Waiting for PV local-pv7m8ql to bind to PVC pvc-c84hb May 15 01:51:48.894: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-c84hb] to have phase Bound May 15 01:51:48.897: INFO: PersistentVolumeClaim pvc-c84hb found but phase is Pending instead of Bound. May 15 01:51:50.901: INFO: PersistentVolumeClaim pvc-c84hb found but phase is Pending instead of Bound. May 15 01:51:52.907: INFO: PersistentVolumeClaim pvc-c84hb found but phase is Pending instead of Bound. May 15 01:51:54.912: INFO: PersistentVolumeClaim pvc-c84hb found but phase is Pending instead of Bound. May 15 01:51:56.915: INFO: PersistentVolumeClaim pvc-c84hb found but phase is Pending instead of Bound. May 15 01:51:58.918: INFO: PersistentVolumeClaim pvc-c84hb found and phase=Bound (10.024360464s) May 15 01:51:58.918: INFO: Waiting up to 3m0s for PersistentVolume local-pv7m8ql to have phase Bound May 15 01:51:58.921: INFO: PersistentVolume local-pv7m8ql found and phase=Bound (2.374872ms) [BeforeEach] Set fsGroup for local volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:257 [It] should set different fsGroup for second pod if first pod is deleted /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:282 May 15 01:51:58.926: INFO: Disabled temporarily, reopen after #73168 is fixed [AfterEach] [Volume type: blockfswithoutformat] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:51:58.928: INFO: Deleting PersistentVolumeClaim "pvc-c84hb" May 15 01:51:58.933: INFO: Deleting PersistentVolume "local-pv7m8ql" May 15 01:51:58.937: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c E2E_LOOP_DEV=$(losetup | grep /tmp/local-volume-test-1b2c98a3-07c1-4c69-9160-dbada5bb7a62/file | awk '{ print $1 }') 2>&1 > /dev/null && echo ${E2E_LOOP_DEV}] Namespace:persistent-local-volumes-test-5616 PodName:hostexec-node1-kstrk ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:51:58.937: INFO: >>> kubeConfig: /root/.kube/config STEP: Tear down block device "/dev/loop2" on node "node1" at path /tmp/local-volume-test-1b2c98a3-07c1-4c69-9160-dbada5bb7a62/file May 15 01:51:59.064: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c losetup -d /dev/loop2] Namespace:persistent-local-volumes-test-5616 PodName:hostexec-node1-kstrk ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:51:59.064: INFO: >>> kubeConfig: /root/.kube/config STEP: Removing the test directory /tmp/local-volume-test-1b2c98a3-07c1-4c69-9160-dbada5bb7a62 May 15 01:51:59.198: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-1b2c98a3-07c1-4c69-9160-dbada5bb7a62] Namespace:persistent-local-volumes-test-5616 PodName:hostexec-node1-kstrk ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:51:59.198: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:51:59.377: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-5616" for this suite. S [SKIPPING] [14.926 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: blockfswithoutformat] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 Set fsGroup for local volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:256 should set different fsGroup for second pod if first pod is deleted [It] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:282 Disabled temporarily, reopen after #73168 is fixed /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:283 ------------------------------ SSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:49:51.248: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: dir] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes May 15 01:49:55.300: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-4538903f-6dad-4bd8-9b3f-a5a3b3a86d75] Namespace:persistent-local-volumes-test-5409 PodName:hostexec-node1-g9jvw ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:49:55.300: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:49:55.429: INFO: Creating a PV followed by a PVC May 15 01:49:55.438: INFO: Waiting for PV local-pv49whp to bind to PVC pvc-tbjsh May 15 01:49:55.438: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-tbjsh] to have phase Bound May 15 01:49:55.440: INFO: PersistentVolumeClaim pvc-tbjsh found but phase is Pending instead of Bound. May 15 01:49:57.443: INFO: PersistentVolumeClaim pvc-tbjsh found but phase is Pending instead of Bound. May 15 01:49:59.448: INFO: PersistentVolumeClaim pvc-tbjsh found and phase=Bound (4.009983285s) May 15 01:49:59.448: INFO: Waiting up to 3m0s for PersistentVolume local-pv49whp to have phase Bound May 15 01:49:59.451: INFO: PersistentVolume local-pv49whp found and phase=Bound (3.044992ms) [It] should be able to write from pod1 and read from pod2 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:245 STEP: Creating pod1 to write to the PV STEP: Creating a pod May 15 01:51:59.477: FAIL: Unexpected error: <*errors.errorString | 0xc004fb2640>: { s: "pod \"pod-e94cee9f-0eca-47df-9ce8-1d446f676d35\" is not Running: timed out waiting for the condition", } pod "pod-e94cee9f-0eca-47df-9ce8-1d446f676d35" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage.twoPodsReadWriteTest(0xc002176160, 0xc003560580, 0xc003afe660) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:727 +0xaf k8s.io/kubernetes/test/e2e/storage.glob..func20.2.4.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:246 +0x45 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc003230480) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc003230480) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc003230480, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [Volume type: dir] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:51:59.479: INFO: Deleting PersistentVolumeClaim "pvc-tbjsh" May 15 01:51:59.483: INFO: Deleting PersistentVolume "local-pv49whp" STEP: Removing the test directory May 15 01:51:59.487: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-4538903f-6dad-4bd8-9b3f-a5a3b3a86d75] Namespace:persistent-local-volumes-test-5409 PodName:hostexec-node1-g9jvw ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:51:59.487: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "persistent-local-volumes-test-5409". STEP: Found 13 events. May 15 01:51:59.639: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for hostexec-node1-g9jvw: { } Scheduled: Successfully assigned persistent-local-volumes-test-5409/hostexec-node1-g9jvw to node1 May 15 01:51:59.639: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-e94cee9f-0eca-47df-9ce8-1d446f676d35: { } Scheduled: Successfully assigned persistent-local-volumes-test-5409/pod-e94cee9f-0eca-47df-9ce8-1d446f676d35 to node1 May 15 01:51:59.639: INFO: At 2021-05-15 01:49:51 +0000 UTC - event for hostexec-node1-g9jvw: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 15 01:51:59.639: INFO: At 2021-05-15 01:49:53 +0000 UTC - event for hostexec-node1-g9jvw: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 1.591991345s May 15 01:51:59.639: INFO: At 2021-05-15 01:49:53 +0000 UTC - event for hostexec-node1-g9jvw: {kubelet node1} Created: Created container agnhost-container May 15 01:51:59.639: INFO: At 2021-05-15 01:49:53 +0000 UTC - event for hostexec-node1-g9jvw: {kubelet node1} Started: Started container agnhost-container May 15 01:51:59.639: INFO: At 2021-05-15 01:49:55 +0000 UTC - event for pvc-tbjsh: {persistentvolume-controller } ProvisioningFailed: no volume plugin matched name: kubernetes.io/no-provisioner May 15 01:51:59.639: INFO: At 2021-05-15 01:50:01 +0000 UTC - event for pod-e94cee9f-0eca-47df-9ce8-1d446f676d35: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:51:59.639: INFO: At 2021-05-15 01:50:01 +0000 UTC - event for pod-e94cee9f-0eca-47df-9ce8-1d446f676d35: {multus } AddedInterface: Add eth0 [10.244.3.172/24] May 15 01:51:59.639: INFO: At 2021-05-15 01:50:02 +0000 UTC - event for pod-e94cee9f-0eca-47df-9ce8-1d446f676d35: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:51:59.639: INFO: At 2021-05-15 01:50:02 +0000 UTC - event for pod-e94cee9f-0eca-47df-9ce8-1d446f676d35: {kubelet node1} Failed: Error: ErrImagePull May 15 01:51:59.639: INFO: At 2021-05-15 01:50:03 +0000 UTC - event for pod-e94cee9f-0eca-47df-9ce8-1d446f676d35: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 15 01:51:59.639: INFO: At 2021-05-15 01:50:03 +0000 UTC - event for pod-e94cee9f-0eca-47df-9ce8-1d446f676d35: {kubelet node1} Failed: Error: ImagePullBackOff May 15 01:51:59.643: INFO: POD NODE PHASE GRACE CONDITIONS May 15 01:51:59.643: INFO: hostexec-node1-g9jvw node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:49:51 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:49:54 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:49:54 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:49:51 +0000 UTC }] May 15 01:51:59.643: INFO: pod-e94cee9f-0eca-47df-9ce8-1d446f676d35 node1 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:49:59 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:49:59 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:49:59 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:49:59 +0000 UTC }] May 15 01:51:59.643: INFO: May 15 01:51:59.647: INFO: Logging node info for node master1 May 15 01:51:59.650: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 159722 0 2021-05-14 19:56:35 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:59 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:59 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:59 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:51:59 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[@ :],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:51:59.650: INFO: Logging kubelet events for node master1 May 15 01:51:59.655: INFO: Logging pods the kubelet thinks is on node master1 May 15 01:51:59.664: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.664: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:51:59.664: INFO: kubernetes-metrics-scraper-678c97765c-fswrn started at 2021-05-15 00:18:49 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.664: INFO: Container kubernetes-metrics-scraper ready: true, restart count 0 May 15 01:51:59.664: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.664: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:51:59.664: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.664: INFO: Container kube-multus ready: true, restart count 1 May 15 01:51:59.664: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 15 01:51:59.664: INFO: Container docker-registry ready: true, restart count 0 May 15 01:51:59.664: INFO: Container nginx ready: true, restart count 0 May 15 01:51:59.664: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.664: INFO: Container nfd-controller ready: true, restart count 0 May 15 01:51:59.664: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.664: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:51:59.664: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:51:59.664: INFO: Init container install-cni ready: true, restart count 0 May 15 01:51:59.664: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:51:59.664: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:51:59.664: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:51:59.664: INFO: Container node-exporter ready: true, restart count 0 May 15 01:51:59.664: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.664: INFO: Container kube-scheduler ready: true, restart count 0 W0515 01:51:59.677352 32 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:51:59.713: INFO: Latency metrics for node master1 May 15 01:51:59.713: INFO: Logging node info for node master2 May 15 01:51:59.716: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 159697 0 2021-05-14 19:57:04 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:56 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:56 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:56 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:51:56 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:51:59.716: INFO: Logging kubelet events for node master2 May 15 01:51:59.720: INFO: Logging pods the kubelet thinks is on node master2 May 15 01:51:59.730: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.730: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:51:59.730: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.730: INFO: Container kube-multus ready: true, restart count 1 May 15 01:51:59.730: INFO: coredns-7677f9bb54-96w24 started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.730: INFO: Container coredns ready: true, restart count 0 May 15 01:51:59.730: INFO: prometheus-operator-5bb8cb9d8f-fqb87 started at 2021-05-15 00:18:49 +0000 UTC (0+2 container statuses recorded) May 15 01:51:59.730: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:51:59.730: INFO: Container prometheus-operator ready: true, restart count 0 May 15 01:51:59.730: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.730: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:51:59.730: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.730: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:51:59.730: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.730: INFO: Container kube-scheduler ready: true, restart count 2 May 15 01:51:59.730: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:51:59.730: INFO: Init container install-cni ready: true, restart count 0 May 15 01:51:59.730: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:51:59.730: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.730: INFO: Container autoscaler ready: true, restart count 2 May 15 01:51:59.730: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:51:59.730: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:51:59.730: INFO: Container node-exporter ready: true, restart count 0 W0515 01:51:59.741760 32 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:51:59.766: INFO: Latency metrics for node master2 May 15 01:51:59.766: INFO: Logging node info for node master3 May 15 01:51:59.768: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 159679 0 2021-05-14 19:57:14 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234767872 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324603904 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:54 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:54 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:54 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:51:54 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:51:59.769: INFO: Logging kubelet events for node master3 May 15 01:51:59.774: INFO: Logging pods the kubelet thinks is on node master3 May 15 01:51:59.781: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.781: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:51:59.781: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.781: INFO: Container kube-proxy ready: true, restart count 1 May 15 01:51:59.781: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:51:59.781: INFO: Init container install-cni ready: true, restart count 0 May 15 01:51:59.781: INFO: Container kube-flannel ready: true, restart count 2 May 15 01:51:59.781: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.781: INFO: Container kube-multus ready: true, restart count 1 May 15 01:51:59.781: INFO: coredns-7677f9bb54-rpj8c started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.781: INFO: Container coredns ready: true, restart count 0 May 15 01:51:59.781: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:51:59.781: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:51:59.781: INFO: Container node-exporter ready: true, restart count 0 May 15 01:51:59.781: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.781: INFO: Container kube-controller-manager ready: true, restart count 3 May 15 01:51:59.781: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.781: INFO: Container kube-scheduler ready: true, restart count 3 W0515 01:51:59.793980 32 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:51:59.820: INFO: Latency metrics for node master3 May 15 01:51:59.820: INFO: Logging node info for node node1 May 15 01:51:59.822: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 159690 0 2021-05-14 19:58:21 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1602":"csi-mock-csi-mock-volumes-1602","csi-mock-csi-mock-volumes-253":"csi-mock-csi-mock-volumes-253","csi-mock-csi-mock-volumes-7734":"csi-mock-csi-mock-volumes-7734","csi-mock-csi-mock-volumes-895":"csi-mock-csi-mock-volumes-895","csi-mock-csi-mock-volumes-9728":"csi-mock-csi-mock-volumes-9728","csi-mock-csi-mock-volumes-9809":"csi-mock-csi-mock-volumes-9809"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:00 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:50:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:51:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:56 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:56 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:56 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:51:56 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[@ :],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:51:59.823: INFO: Logging kubelet events for node node1 May 15 01:51:59.828: INFO: Logging pods the kubelet thinks is on node node1 May 15 01:51:59.843: INFO: prometheus-k8s-0 started at 2021-05-15 00:19:01 +0000 UTC (0+5 container statuses recorded) May 15 01:51:59.843: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 15 01:51:59.843: INFO: Container grafana ready: true, restart count 0 May 15 01:51:59.843: INFO: Container prometheus ready: true, restart count 26 May 15 01:51:59.843: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 15 01:51:59.843: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 15 01:51:59.843: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.843: INFO: Container kube-multus ready: true, restart count 1 May 15 01:51:59.843: INFO: pod-58f129b2-416d-48d8-93b6-2628e4cc65d5 started at 2021-05-15 01:50:35 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.843: INFO: Container write-pod ready: false, restart count 0 May 15 01:51:59.843: INFO: pod-e94cee9f-0eca-47df-9ce8-1d446f676d35 started at 2021-05-15 01:49:59 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.843: INFO: Container write-pod ready: false, restart count 0 May 15 01:51:59.843: INFO: pod-bfba8786-cc42-4bcf-babe-8aa8c1a6f88f started at 2021-05-15 01:51:46 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.843: INFO: Container write-pod ready: false, restart count 0 May 15 01:51:59.843: INFO: pod-6fec197a-289d-4046-b7a7-00d385f2dc10 started at 2021-05-15 01:50:35 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.843: INFO: Container write-pod ready: false, restart count 0 May 15 01:51:59.843: INFO: hostexec-node1-ktptr started at 2021-05-15 01:50:37 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.843: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:51:59.843: INFO: node-exporter-flvqz started at 2021-05-15 00:18:55 +0000 UTC (0+2 container statuses recorded) May 15 01:51:59.844: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:51:59.844: INFO: Container node-exporter ready: true, restart count 0 May 15 01:51:59.844: INFO: cmk-4s6dm started at 2021-05-15 00:18:54 +0000 UTC (0+2 container statuses recorded) May 15 01:51:59.844: INFO: Container nodereport ready: true, restart count 0 May 15 01:51:59.844: INFO: Container reconcile ready: true, restart count 0 May 15 01:51:59.844: INFO: pod-9566f4e9-ff2d-489c-94bf-709e28a35a7d started at 2021-05-15 01:50:43 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.844: INFO: Container write-pod ready: false, restart count 0 May 15 01:51:59.844: INFO: hostexec-node1-g9jvw started at 2021-05-15 01:49:51 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.844: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:51:59.844: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4vxcc started at 2021-05-15 00:19:00 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.844: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:51:59.844: INFO: hostexec-node1-vlzvd started at 2021-05-15 01:51:39 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.844: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:51:59.844: INFO: hostexec-node1-gkk5h started at 2021-05-15 01:51:59 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.844: INFO: Container agnhost-container ready: false, restart count 0 May 15 01:51:59.844: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.844: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:51:59.844: INFO: hostexec-node1-2l9b5 started at 2021-05-15 01:50:29 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.844: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:51:59.844: INFO: hostexec-node1-5sh66 started at 2021-05-15 01:51:44 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.844: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:51:59.844: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.844: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:51:59.844: INFO: hostexec-node1-kstrk started at 2021-05-15 01:51:44 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.844: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:51:59.844: INFO: hostexec-node1-zbxmq started at 2021-05-15 01:50:51 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.844: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:51:59.844: INFO: hostexec-node1-dwvtr started at 2021-05-15 01:50:31 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.844: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:51:59.844: INFO: node-feature-discovery-worker-bw8zg started at 2021-05-15 00:18:56 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.844: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:51:59.844: INFO: collectd-mrzps started at 2021-05-15 00:19:22 +0000 UTC (0+3 container statuses recorded) May 15 01:51:59.844: INFO: Container collectd ready: true, restart count 0 May 15 01:51:59.844: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:51:59.844: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:51:59.844: INFO: pod-11760848-bbb5-4ae1-af82-a7a2a4af6a9c started at 2021-05-15 01:50:59 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.844: INFO: Container write-pod ready: false, restart count 0 May 15 01:51:59.844: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:51:59.844: INFO: Init container install-cni ready: true, restart count 0 May 15 01:51:59.844: INFO: Container kube-flannel ready: true, restart count 1 W0515 01:51:59.858321 32 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:51:59.963: INFO: Latency metrics for node node1 May 15 01:51:59.963: INFO: Logging node info for node node2 May 15 01:51:59.966: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 159641 0 2021-05-14 19:58:22 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1663":"csi-mock-csi-mock-volumes-1663","csi-mock-csi-mock-volumes-3052":"csi-mock-csi-mock-volumes-3052","csi-mock-csi-mock-volumes-5678":"csi-mock-csi-mock-volumes-5678","csi-mock-csi-mock-volumes-8760":"csi-mock-csi-mock-volumes-8760","csi-mock-csi-mock-volumes-9624":"csi-mock-csi-mock-volumes-9624"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:46 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:44:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:46:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:io.kubernetes.storage.mock/node":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:50 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:50 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:51:50 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:51:50 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:51:59.967: INFO: Logging kubelet events for node node2 May 15 01:51:59.971: INFO: Logging pods the kubelet thinks is on node node2 May 15 01:51:59.980: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 15 01:51:59.980: INFO: Container tas-controller ready: true, restart count 0 May 15 01:51:59.980: INFO: Container tas-extender ready: true, restart count 0 May 15 01:51:59.980: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 15 01:51:59.980: INFO: Container collectd ready: true, restart count 0 May 15 01:51:59.980: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:51:59.980: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:51:59.980: INFO: pod-48eb4d4e-872f-434e-bb95-a838c2188d36 started at 2021-05-15 01:51:59 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.980: INFO: Container test-container ready: false, restart count 0 May 15 01:51:59.980: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.980: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:51:59.980: INFO: pod-subpath-test-configmap-lp9m started at 2021-05-15 01:50:05 +0000 UTC (1+2 container statuses recorded) May 15 01:51:59.980: INFO: Init container init-volume-configmap-lp9m ready: false, restart count 0 May 15 01:51:59.980: INFO: Container test-container-subpath-configmap-lp9m ready: false, restart count 0 May 15 01:51:59.980: INFO: Container test-container-volume-configmap-lp9m ready: false, restart count 0 May 15 01:51:59.980: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.980: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:51:59.980: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 15 01:51:59.980: INFO: Container nodereport ready: true, restart count 0 May 15 01:51:59.980: INFO: Container reconcile ready: true, restart count 0 May 15 01:51:59.980: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.980: INFO: Container cmk-webhook ready: true, restart count 0 May 15 01:51:59.980: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 15 01:51:59.980: INFO: Container discover ready: false, restart count 0 May 15 01:51:59.980: INFO: Container init ready: false, restart count 0 May 15 01:51:59.980: INFO: Container install ready: false, restart count 0 May 15 01:51:59.980: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.980: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:51:59.980: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.980: INFO: Container kube-multus ready: true, restart count 1 May 15 01:51:59.980: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.980: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 15 01:51:59.980: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:51:59.980: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:51:59.980: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:51:59.980: INFO: Init container install-cni ready: true, restart count 1 May 15 01:51:59.980: INFO: Container kube-flannel ready: true, restart count 4 May 15 01:51:59.980: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:51:59.980: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:51:59.980: INFO: Container node-exporter ready: true, restart count 0 W0515 01:51:59.993483 32 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:52:00.355: INFO: Latency metrics for node node2 May 15 01:52:00.355: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-5409" for this suite. • Failure [129.121 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: dir] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 Two pods mounting a local volume at the same time /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:244 should be able to write from pod1 and read from pod2 [It] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:245 May 15 01:51:59.477: Unexpected error: <*errors.errorString | 0xc004fb2640>: { s: "pod \"pod-e94cee9f-0eca-47df-9ce8-1d446f676d35\" is not Running: timed out waiting for the condition", } pod "pod-e94cee9f-0eca-47df-9ce8-1d446f676d35" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:727 ------------------------------ {"msg":"FAILED [sig-storage] PersistentVolumes-local [Volume type: dir] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2","total":-1,"completed":3,"skipped":115,"failed":3,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir-link] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir-link] One pod requesting one prebound PVC should be able to mount volume and write from pod1","[sig-storage] PersistentVolumes-local [Volume type: dir] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2"]} SSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] EmptyDir volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:51:59.399: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename emptydir STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] when FSGroup is specified [LinuxOnly] [NodeFeature:FSGroup] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/empty_dir.go:47 [It] new files should be created with FSGroup ownership when container is non-root /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/empty_dir.go:56 STEP: Creating a pod to test emptydir 0644 on tmpfs May 15 01:51:59.432: INFO: Waiting up to 5m0s for pod "pod-48eb4d4e-872f-434e-bb95-a838c2188d36" in namespace "emptydir-7979" to be "Succeeded or Failed" May 15 01:51:59.435: INFO: Pod "pod-48eb4d4e-872f-434e-bb95-a838c2188d36": Phase="Pending", Reason="", readiness=false. Elapsed: 2.992353ms May 15 01:52:01.439: INFO: Pod "pod-48eb4d4e-872f-434e-bb95-a838c2188d36": Phase="Pending", Reason="", readiness=false. Elapsed: 2.006564331s May 15 01:52:03.442: INFO: Pod "pod-48eb4d4e-872f-434e-bb95-a838c2188d36": Phase="Pending", Reason="", readiness=false. Elapsed: 4.009602976s May 15 01:52:05.448: INFO: Pod "pod-48eb4d4e-872f-434e-bb95-a838c2188d36": Phase="Succeeded", Reason="", readiness=false. Elapsed: 6.015814508s STEP: Saw pod success May 15 01:52:05.448: INFO: Pod "pod-48eb4d4e-872f-434e-bb95-a838c2188d36" satisfied condition "Succeeded or Failed" May 15 01:52:05.450: INFO: Trying to get logs from node node2 pod pod-48eb4d4e-872f-434e-bb95-a838c2188d36 container test-container: STEP: delete the pod May 15 01:52:05.467: INFO: Waiting for pod pod-48eb4d4e-872f-434e-bb95-a838c2188d36 to disappear May 15 01:52:05.469: INFO: Pod pod-48eb4d4e-872f-434e-bb95-a838c2188d36 no longer exists [AfterEach] [sig-storage] EmptyDir volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:52:05.469: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "emptydir-7979" for this suite. • [SLOW TEST:6.077 seconds] [sig-storage] EmptyDir volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/empty_dir.go:42 when FSGroup is specified [LinuxOnly] [NodeFeature:FSGroup] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/empty_dir.go:45 new files should be created with FSGroup ownership when container is non-root /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/empty_dir.go:56 ------------------------------ {"msg":"PASSED [sig-storage] EmptyDir volumes when FSGroup is specified [LinuxOnly] [NodeFeature:FSGroup] new files should be created with FSGroup ownership when container is non-root","total":-1,"completed":3,"skipped":350,"failed":3,"failures":["[sig-storage] PersistentVolumes-local [Volume type: blockfswithoutformat] One pod requesting one prebound PVC should be able to mount volume and read from pod1","[sig-storage] PersistentVolumes-local [Volume type: blockfswithformat] One pod requesting one prebound PVC should be able to mount volume and read from pod1","[sig-storage] PersistentVolumes-local [Volume type: tmpfs] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2"]} SSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:51:15.045: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename csi-mock-volumes STEP: Waiting for a default service account to be provisioned in namespace [It] exhausted, late binding, no topology /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:805 STEP: Building a driver namespace object, basename csi-mock-volumes-895 STEP: Waiting for a default service account to be provisioned in namespace STEP: deploying csi mock driver May 15 01:51:15.130: INFO: creating *v1.ServiceAccount: csi-mock-volumes-895-8409/csi-attacher May 15 01:51:15.133: INFO: creating *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-895 May 15 01:51:15.133: INFO: Define cluster role external-attacher-runner-csi-mock-volumes-895 May 15 01:51:15.136: INFO: creating *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-895 May 15 01:51:15.139: INFO: creating *v1.Role: csi-mock-volumes-895-8409/external-attacher-cfg-csi-mock-volumes-895 May 15 01:51:15.142: INFO: creating *v1.RoleBinding: csi-mock-volumes-895-8409/csi-attacher-role-cfg May 15 01:51:15.144: INFO: creating *v1.ServiceAccount: csi-mock-volumes-895-8409/csi-provisioner May 15 01:51:15.147: INFO: creating *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-895 May 15 01:51:15.147: INFO: Define cluster role external-provisioner-runner-csi-mock-volumes-895 May 15 01:51:15.149: INFO: creating *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-895 May 15 01:51:15.152: INFO: creating *v1.Role: csi-mock-volumes-895-8409/external-provisioner-cfg-csi-mock-volumes-895 May 15 01:51:15.155: INFO: creating *v1.RoleBinding: csi-mock-volumes-895-8409/csi-provisioner-role-cfg May 15 01:51:15.157: INFO: creating *v1.ServiceAccount: csi-mock-volumes-895-8409/csi-resizer May 15 01:51:15.160: INFO: creating *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-895 May 15 01:51:15.160: INFO: Define cluster role external-resizer-runner-csi-mock-volumes-895 May 15 01:51:15.163: INFO: creating *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-895 May 15 01:51:15.166: INFO: creating *v1.Role: csi-mock-volumes-895-8409/external-resizer-cfg-csi-mock-volumes-895 May 15 01:51:15.168: INFO: creating *v1.RoleBinding: csi-mock-volumes-895-8409/csi-resizer-role-cfg May 15 01:51:15.171: INFO: creating *v1.ServiceAccount: csi-mock-volumes-895-8409/csi-mock May 15 01:51:15.173: INFO: creating *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-895 May 15 01:51:15.175: INFO: creating *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-895 May 15 01:51:15.178: INFO: creating *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-895 May 15 01:51:15.180: INFO: creating *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-895 May 15 01:51:15.183: INFO: creating *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-895 May 15 01:51:15.185: INFO: creating *v1.StorageClass: csi-mock-sc-csi-mock-volumes-895 May 15 01:51:15.187: INFO: creating *v1.StatefulSet: csi-mock-volumes-895-8409/csi-mockplugin May 15 01:51:15.192: INFO: creating *v1.CSIDriver: csi-mock-csi-mock-volumes-895 May 15 01:51:15.195: INFO: waiting up to 4m0s for CSIDriver "csi-mock-csi-mock-volumes-895" May 15 01:51:15.197: INFO: waiting for CSIDriver csi-mock-csi-mock-volumes-895 to register on node node1 STEP: Creating pod May 15 01:51:20.215: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil May 15 01:51:26.237: INFO: Deleting pod "pvc-volume-tester-9b8r9" in namespace "csi-mock-volumes-895" May 15 01:51:26.243: INFO: Wait up to 5m0s for pod "pvc-volume-tester-9b8r9" to be fully deleted STEP: Checking PVC events May 15 01:51:41.270: INFO: PVC event ADDED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-79ds9", GenerateName:"pvc-", Namespace:"csi-mock-volumes-895", SelfLink:"/api/v1/namespaces/csi-mock-volumes-895/persistentvolumeclaims/pvc-79ds9", UID:"ea7be7e3-5378-4d83-ae4a-0241fb693400", ResourceVersion:"159160", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63756640280, loc:(*time.Location)(0x770c940)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"e2e.test", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc0030fa280), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0030fa2a0)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"", StorageClassName:(*string)(0xc0011fa200), VolumeMode:(*v1.PersistentVolumeMode)(0xc0011fa210), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Pending", AccessModes:[]v1.PersistentVolumeAccessMode(nil), Capacity:v1.ResourceList(nil), Conditions:[]v1.PersistentVolumeClaimCondition(nil)}} May 15 01:51:41.270: INFO: PVC event MODIFIED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-79ds9", GenerateName:"pvc-", Namespace:"csi-mock-volumes-895", SelfLink:"/api/v1/namespaces/csi-mock-volumes-895/persistentvolumeclaims/pvc-79ds9", UID:"ea7be7e3-5378-4d83-ae4a-0241fb693400", ResourceVersion:"159163", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63756640280, loc:(*time.Location)(0x770c940)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string{"volume.kubernetes.io/selected-node":"node1"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"e2e.test", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc0030fa6c0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0030fa6e0)}, v1.ManagedFieldsEntry{Manager:"kube-scheduler", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc0030fa700), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0030fa720)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"", StorageClassName:(*string)(0xc0011fa3a0), VolumeMode:(*v1.PersistentVolumeMode)(0xc0011fa3b0), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Pending", AccessModes:[]v1.PersistentVolumeAccessMode(nil), Capacity:v1.ResourceList(nil), Conditions:[]v1.PersistentVolumeClaimCondition(nil)}} May 15 01:51:41.270: INFO: PVC event MODIFIED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-79ds9", GenerateName:"pvc-", Namespace:"csi-mock-volumes-895", SelfLink:"/api/v1/namespaces/csi-mock-volumes-895/persistentvolumeclaims/pvc-79ds9", UID:"ea7be7e3-5378-4d83-ae4a-0241fb693400", ResourceVersion:"159164", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63756640280, loc:(*time.Location)(0x770c940)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string{"volume.beta.kubernetes.io/storage-provisioner":"csi-mock-csi-mock-volumes-895", "volume.kubernetes.io/selected-node":"node1"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"e2e.test", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc0030faac0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0030faae0)}, v1.ManagedFieldsEntry{Manager:"kube-controller-manager", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc0030fab00), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0030fab20)}, v1.ManagedFieldsEntry{Manager:"kube-scheduler", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc0030fab40), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0030fab60)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"", StorageClassName:(*string)(0xc0011fa5b0), VolumeMode:(*v1.PersistentVolumeMode)(0xc0011fa5c0), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Pending", AccessModes:[]v1.PersistentVolumeAccessMode(nil), Capacity:v1.ResourceList(nil), Conditions:[]v1.PersistentVolumeClaimCondition(nil)}} May 15 01:51:41.270: INFO: PVC event MODIFIED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-79ds9", GenerateName:"pvc-", Namespace:"csi-mock-volumes-895", SelfLink:"/api/v1/namespaces/csi-mock-volumes-895/persistentvolumeclaims/pvc-79ds9", UID:"ea7be7e3-5378-4d83-ae4a-0241fb693400", ResourceVersion:"159172", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63756640280, loc:(*time.Location)(0x770c940)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string{"pv.kubernetes.io/bind-completed":"yes", "pv.kubernetes.io/bound-by-controller":"yes", "volume.beta.kubernetes.io/storage-provisioner":"csi-mock-csi-mock-volumes-895", "volume.kubernetes.io/selected-node":"node1"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"e2e.test", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc003e0d3a0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc003e0d3c0)}, v1.ManagedFieldsEntry{Manager:"kube-controller-manager", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc003e0d3e0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc003e0d400)}, v1.ManagedFieldsEntry{Manager:"kube-scheduler", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc003e0d420), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc003e0d440)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"pvc-ea7be7e3-5378-4d83-ae4a-0241fb693400", StorageClassName:(*string)(0xc001dc0d30), VolumeMode:(*v1.PersistentVolumeMode)(0xc001dc0d40), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Pending", AccessModes:[]v1.PersistentVolumeAccessMode(nil), Capacity:v1.ResourceList(nil), Conditions:[]v1.PersistentVolumeClaimCondition(nil)}} May 15 01:51:41.270: INFO: PVC event MODIFIED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-79ds9", GenerateName:"pvc-", Namespace:"csi-mock-volumes-895", SelfLink:"/api/v1/namespaces/csi-mock-volumes-895/persistentvolumeclaims/pvc-79ds9", UID:"ea7be7e3-5378-4d83-ae4a-0241fb693400", ResourceVersion:"159173", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63756640280, loc:(*time.Location)(0x770c940)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string{"pv.kubernetes.io/bind-completed":"yes", "pv.kubernetes.io/bound-by-controller":"yes", "volume.beta.kubernetes.io/storage-provisioner":"csi-mock-csi-mock-volumes-895", "volume.kubernetes.io/selected-node":"node1"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"e2e.test", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc003e0d480), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc003e0d4a0)}, v1.ManagedFieldsEntry{Manager:"kube-controller-manager", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc003e0d4c0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc003e0d4e0)}, v1.ManagedFieldsEntry{Manager:"kube-scheduler", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc003e0d500), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc003e0d520)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"pvc-ea7be7e3-5378-4d83-ae4a-0241fb693400", StorageClassName:(*string)(0xc001dc0d70), VolumeMode:(*v1.PersistentVolumeMode)(0xc001dc0d80), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Bound", AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Capacity:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}, Conditions:[]v1.PersistentVolumeClaimCondition(nil)}} May 15 01:51:41.270: INFO: PVC event MODIFIED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-79ds9", GenerateName:"pvc-", Namespace:"csi-mock-volumes-895", SelfLink:"/api/v1/namespaces/csi-mock-volumes-895/persistentvolumeclaims/pvc-79ds9", UID:"ea7be7e3-5378-4d83-ae4a-0241fb693400", ResourceVersion:"159407", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63756640280, loc:(*time.Location)(0x770c940)}}, DeletionTimestamp:(*v1.Time)(0xc00446c020), DeletionGracePeriodSeconds:(*int64)(0xc00524fb58), Labels:map[string]string(nil), Annotations:map[string]string{"pv.kubernetes.io/bind-completed":"yes", "pv.kubernetes.io/bound-by-controller":"yes", "volume.beta.kubernetes.io/storage-provisioner":"csi-mock-csi-mock-volumes-895", "volume.kubernetes.io/selected-node":"node1"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"e2e.test", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc00446c040), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc00446c060)}, v1.ManagedFieldsEntry{Manager:"kube-controller-manager", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc00446c080), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc00446c0a0)}, v1.ManagedFieldsEntry{Manager:"kube-scheduler", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc00446c0c0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc00446c0e0)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"pvc-ea7be7e3-5378-4d83-ae4a-0241fb693400", StorageClassName:(*string)(0xc001ba0160), VolumeMode:(*v1.PersistentVolumeMode)(0xc001ba0170), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Bound", AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Capacity:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}, Conditions:[]v1.PersistentVolumeClaimCondition(nil)}} May 15 01:51:41.271: INFO: PVC event DELETED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-79ds9", GenerateName:"pvc-", Namespace:"csi-mock-volumes-895", SelfLink:"/api/v1/namespaces/csi-mock-volumes-895/persistentvolumeclaims/pvc-79ds9", UID:"ea7be7e3-5378-4d83-ae4a-0241fb693400", ResourceVersion:"159408", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63756640280, loc:(*time.Location)(0x770c940)}}, DeletionTimestamp:(*v1.Time)(0xc00446c120), DeletionGracePeriodSeconds:(*int64)(0xc00524fc28), Labels:map[string]string(nil), Annotations:map[string]string{"pv.kubernetes.io/bind-completed":"yes", "pv.kubernetes.io/bound-by-controller":"yes", "volume.beta.kubernetes.io/storage-provisioner":"csi-mock-csi-mock-volumes-895", "volume.kubernetes.io/selected-node":"node1"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"e2e.test", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc00446c140), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc00446c160)}, v1.ManagedFieldsEntry{Manager:"kube-controller-manager", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc00446c180), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc00446c1a0)}, v1.ManagedFieldsEntry{Manager:"kube-scheduler", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc00446c1c0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc00446c1e0)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"pvc-ea7be7e3-5378-4d83-ae4a-0241fb693400", StorageClassName:(*string)(0xc001ba01b0), VolumeMode:(*v1.PersistentVolumeMode)(0xc001ba01c0), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Bound", AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Capacity:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}, Conditions:[]v1.PersistentVolumeClaimCondition(nil)}} STEP: Deleting pod pvc-volume-tester-9b8r9 May 15 01:51:41.271: INFO: Deleting pod "pvc-volume-tester-9b8r9" in namespace "csi-mock-volumes-895" STEP: Deleting claim pvc-79ds9 STEP: Deleting storageclass csi-mock-volumes-895-sc STEP: Cleaning up resources STEP: deleting the test namespace: csi-mock-volumes-895 STEP: Waiting for namespaces [csi-mock-volumes-895] to vanish STEP: uninstalling csi mock driver May 15 01:51:47.293: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-895-8409/csi-attacher May 15 01:51:47.298: INFO: deleting *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-895 May 15 01:51:47.302: INFO: deleting *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-895 May 15 01:51:47.306: INFO: deleting *v1.Role: csi-mock-volumes-895-8409/external-attacher-cfg-csi-mock-volumes-895 May 15 01:51:47.310: INFO: deleting *v1.RoleBinding: csi-mock-volumes-895-8409/csi-attacher-role-cfg May 15 01:51:47.314: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-895-8409/csi-provisioner May 15 01:51:47.321: INFO: deleting *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-895 May 15 01:51:47.332: INFO: deleting *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-895 May 15 01:51:47.344: INFO: deleting *v1.Role: csi-mock-volumes-895-8409/external-provisioner-cfg-csi-mock-volumes-895 May 15 01:51:47.352: INFO: deleting *v1.RoleBinding: csi-mock-volumes-895-8409/csi-provisioner-role-cfg May 15 01:51:47.357: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-895-8409/csi-resizer May 15 01:51:47.362: INFO: deleting *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-895 May 15 01:51:47.367: INFO: deleting *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-895 May 15 01:51:47.371: INFO: deleting *v1.Role: csi-mock-volumes-895-8409/external-resizer-cfg-csi-mock-volumes-895 May 15 01:51:47.375: INFO: deleting *v1.RoleBinding: csi-mock-volumes-895-8409/csi-resizer-role-cfg May 15 01:51:47.379: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-895-8409/csi-mock May 15 01:51:47.384: INFO: deleting *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-895 May 15 01:51:47.388: INFO: deleting *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-895 May 15 01:51:47.393: INFO: deleting *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-895 May 15 01:51:47.397: INFO: deleting *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-895 May 15 01:51:47.401: INFO: deleting *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-895 May 15 01:51:47.405: INFO: deleting *v1.StorageClass: csi-mock-sc-csi-mock-volumes-895 May 15 01:51:47.409: INFO: deleting *v1.StatefulSet: csi-mock-volumes-895-8409/csi-mockplugin May 15 01:51:47.413: INFO: deleting *v1.CSIDriver: csi-mock-csi-mock-volumes-895 STEP: deleting the driver namespace: csi-mock-volumes-895-8409 STEP: Waiting for namespaces [csi-mock-volumes-895-8409] to vanish [AfterEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:52:15.429: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "csi-mock-volumes-895" for this suite. May 15 01:52:15.434: INFO: Namespace csi-mock-volumes-895 was already deleted STEP: Destroying namespace "csi-mock-volumes-895-8409" for this suite. May 15 01:52:15.437: INFO: Namespace csi-mock-volumes-895-8409 was already deleted • [SLOW TEST:60.392 seconds] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 storage capacity /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:752 exhausted, late binding, no topology /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:805 ------------------------------ {"msg":"PASSED [sig-storage] CSI mock volume storage capacity exhausted, late binding, no topology","total":-1,"completed":6,"skipped":855,"failed":3,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir] One pod requesting one prebound PVC should be able to mount volume and write from pod1","[sig-storage] PersistentVolumes-local [Volume type: tmpfs] One pod requesting one prebound PVC should be able to mount volume and read from pod1","[sig-storage] PersistentVolumes-local [Volume type: dir-link-bindmounted] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2"]} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:51:59.272: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] Pod with node different from PV's NodeAffinity /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:345 STEP: Initializing test volumes May 15 01:52:01.319: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-7abf5377-47a5-4e6e-8ab6-17a379e10140] Namespace:persistent-local-volumes-test-1879 PodName:hostexec-node1-gkk5h ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:52:01.319: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:52:01.458: INFO: Creating a PV followed by a PVC May 15 01:52:01.464: INFO: Waiting for PV local-pvsm926 to bind to PVC pvc-jt7sm May 15 01:52:01.464: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-jt7sm] to have phase Bound May 15 01:52:01.466: INFO: PersistentVolumeClaim pvc-jt7sm found but phase is Pending instead of Bound. May 15 01:52:03.469: INFO: PersistentVolumeClaim pvc-jt7sm found but phase is Pending instead of Bound. May 15 01:52:05.472: INFO: PersistentVolumeClaim pvc-jt7sm found but phase is Pending instead of Bound. May 15 01:52:07.477: INFO: PersistentVolumeClaim pvc-jt7sm found but phase is Pending instead of Bound. May 15 01:52:09.480: INFO: PersistentVolumeClaim pvc-jt7sm found but phase is Pending instead of Bound. May 15 01:52:11.483: INFO: PersistentVolumeClaim pvc-jt7sm found but phase is Pending instead of Bound. May 15 01:52:13.485: INFO: PersistentVolumeClaim pvc-jt7sm found and phase=Bound (12.021062052s) May 15 01:52:13.485: INFO: Waiting up to 3m0s for PersistentVolume local-pvsm926 to have phase Bound May 15 01:52:13.488: INFO: PersistentVolume local-pvsm926 found and phase=Bound (2.672246ms) [It] should fail scheduling due to different NodeSelector /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:365 STEP: local-volume-type: dir STEP: Initializing test volumes May 15 01:52:13.492: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-ccb91261-613d-4e8d-9ec8-f9a98c2885e7] Namespace:persistent-local-volumes-test-1879 PodName:hostexec-node1-gkk5h ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:52:13.492: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:52:13.620: INFO: Creating a PV followed by a PVC May 15 01:52:13.630: INFO: Waiting for PV local-pv4gvcz to bind to PVC pvc-wh2lk May 15 01:52:13.630: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-wh2lk] to have phase Bound May 15 01:52:13.633: INFO: PersistentVolumeClaim pvc-wh2lk found but phase is Pending instead of Bound. May 15 01:52:15.635: INFO: PersistentVolumeClaim pvc-wh2lk found and phase=Bound (2.004842928s) May 15 01:52:15.635: INFO: Waiting up to 3m0s for PersistentVolume local-pv4gvcz to have phase Bound May 15 01:52:15.637: INFO: PersistentVolume local-pv4gvcz found and phase=Bound (2.079842ms) May 15 01:52:15.652: INFO: Waiting up to 5m0s for pod "pod-471098d7-ea77-4b17-b09e-fc2824dc3b62" in namespace "persistent-local-volumes-test-1879" to be "Unschedulable" May 15 01:52:15.654: INFO: Pod "pod-471098d7-ea77-4b17-b09e-fc2824dc3b62": Phase="Pending", Reason="", readiness=false. Elapsed: 1.78031ms May 15 01:52:17.657: INFO: Pod "pod-471098d7-ea77-4b17-b09e-fc2824dc3b62": Phase="Pending", Reason="", readiness=false. Elapsed: 2.005038701s May 15 01:52:17.657: INFO: Pod "pod-471098d7-ea77-4b17-b09e-fc2824dc3b62" satisfied condition "Unschedulable" [AfterEach] Pod with node different from PV's NodeAffinity /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:356 STEP: Cleaning up PVC and PV May 15 01:52:17.657: INFO: Deleting PersistentVolumeClaim "pvc-jt7sm" May 15 01:52:17.661: INFO: Deleting PersistentVolume "local-pvsm926" STEP: Removing the test directory May 15 01:52:17.665: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-7abf5377-47a5-4e6e-8ab6-17a379e10140] Namespace:persistent-local-volumes-test-1879 PodName:hostexec-node1-gkk5h ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:52:17.665: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:52:17.813: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-1879" for this suite. • [SLOW TEST:18.549 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 Pod with node different from PV's NodeAffinity /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:339 should fail scheduling due to different NodeSelector /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:365 ------------------------------ {"msg":"PASSED [sig-storage] PersistentVolumes-local Pod with node different from PV's NodeAffinity should fail scheduling due to different NodeSelector","total":-1,"completed":4,"skipped":631,"failed":2,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] One pod requesting one prebound PVC should be able to mount volume and write from pod1","[sig-storage] PersistentVolumes-local [Volume type: blockfswithformat] One pod requesting one prebound PVC should be able to mount volume and write from pod1"]} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] Projected downwardAPI /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:52:15.645: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename projected STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] Projected downwardAPI /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/projected_downwardapi.go:42 [It] should provide podname as non-root with fsgroup [LinuxOnly] [NodeFeature:FSGroup] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/projected_downwardapi.go:92 STEP: Creating a pod to test downward API volume plugin May 15 01:52:15.677: INFO: Waiting up to 5m0s for pod "metadata-volume-21e3892f-4ea0-498c-8855-4d88467d08c5" in namespace "projected-8959" to be "Succeeded or Failed" May 15 01:52:15.679: INFO: Pod "metadata-volume-21e3892f-4ea0-498c-8855-4d88467d08c5": Phase="Pending", Reason="", readiness=false. Elapsed: 2.653995ms May 15 01:52:17.683: INFO: Pod "metadata-volume-21e3892f-4ea0-498c-8855-4d88467d08c5": Phase="Pending", Reason="", readiness=false. Elapsed: 2.005856338s May 15 01:52:19.685: INFO: Pod "metadata-volume-21e3892f-4ea0-498c-8855-4d88467d08c5": Phase="Succeeded", Reason="", readiness=false. Elapsed: 4.008683362s STEP: Saw pod success May 15 01:52:19.685: INFO: Pod "metadata-volume-21e3892f-4ea0-498c-8855-4d88467d08c5" satisfied condition "Succeeded or Failed" May 15 01:52:19.688: INFO: Trying to get logs from node node2 pod metadata-volume-21e3892f-4ea0-498c-8855-4d88467d08c5 container client-container: STEP: delete the pod May 15 01:52:19.706: INFO: Waiting for pod metadata-volume-21e3892f-4ea0-498c-8855-4d88467d08c5 to disappear May 15 01:52:19.709: INFO: Pod metadata-volume-21e3892f-4ea0-498c-8855-4d88467d08c5 no longer exists [AfterEach] [sig-storage] Projected downwardAPI /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:52:19.709: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "projected-8959" for this suite. • ------------------------------ {"msg":"PASSED [sig-storage] Projected downwardAPI should provide podname as non-root with fsgroup [LinuxOnly] [NodeFeature:FSGroup]","total":-1,"completed":7,"skipped":969,"failed":3,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir] One pod requesting one prebound PVC should be able to mount volume and write from pod1","[sig-storage] PersistentVolumes-local [Volume type: tmpfs] One pod requesting one prebound PVC should be able to mount volume and read from pod1","[sig-storage] PersistentVolumes-local [Volume type: dir-link-bindmounted] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2"]} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] Mounted volume expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:52:19.788: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename mounted-volume-expand STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] Mounted volume expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/mounted_volume_resize.go:59 May 15 01:52:19.808: INFO: Only supported for providers [aws gce] (not skeleton) [AfterEach] [sig-storage] Mounted volume expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:52:19.810: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "mounted-volume-expand-1238" for this suite. [AfterEach] [sig-storage] Mounted volume expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/mounted_volume_resize.go:104 May 15 01:52:19.818: INFO: AfterEach: Cleaning up resources for mounted volume resize S [SKIPPING] in Spec Setup (BeforeEach) [0.030 seconds] [sig-storage] Mounted volume expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 Should verify mounted devices can be resized [BeforeEach] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/mounted_volume_resize.go:116 Only supported for providers [aws gce] (not skeleton) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/mounted_volume_resize.go:60 ------------------------------ SSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:52:18.007: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] Pod with node different from PV's NodeAffinity /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:345 STEP: Initializing test volumes May 15 01:52:20.051: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-0eb1e386-d168-4ca3-baa7-f6a17212ee9a] Namespace:persistent-local-volumes-test-8279 PodName:hostexec-node1-xb8p5 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:52:20.051: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:52:20.158: INFO: Creating a PV followed by a PVC May 15 01:52:20.166: INFO: Waiting for PV local-pvk98lk to bind to PVC pvc-j72cf May 15 01:52:20.166: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-j72cf] to have phase Bound May 15 01:52:20.168: INFO: PersistentVolumeClaim pvc-j72cf found but phase is Pending instead of Bound. May 15 01:52:22.172: INFO: PersistentVolumeClaim pvc-j72cf found and phase=Bound (2.005845738s) May 15 01:52:22.172: INFO: Waiting up to 3m0s for PersistentVolume local-pvk98lk to have phase Bound May 15 01:52:22.174: INFO: PersistentVolume local-pvk98lk found and phase=Bound (2.542723ms) [It] should fail scheduling due to different NodeAffinity /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:361 STEP: local-volume-type: dir STEP: Initializing test volumes May 15 01:52:22.180: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-39a243e0-a563-40ae-9bf1-aa2a15182ab1] Namespace:persistent-local-volumes-test-8279 PodName:hostexec-node1-xb8p5 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:52:22.180: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:52:22.286: INFO: Creating a PV followed by a PVC May 15 01:52:22.293: INFO: Waiting for PV local-pv58vgw to bind to PVC pvc-b9bw5 May 15 01:52:22.293: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-b9bw5] to have phase Bound May 15 01:52:22.295: INFO: PersistentVolumeClaim pvc-b9bw5 found but phase is Pending instead of Bound. May 15 01:52:24.299: INFO: PersistentVolumeClaim pvc-b9bw5 found and phase=Bound (2.00622106s) May 15 01:52:24.299: INFO: Waiting up to 3m0s for PersistentVolume local-pv58vgw to have phase Bound May 15 01:52:24.301: INFO: PersistentVolume local-pv58vgw found and phase=Bound (2.074239ms) May 15 01:52:24.316: INFO: Waiting up to 5m0s for pod "pod-5e443b34-2e92-4619-9818-fdf83e72b807" in namespace "persistent-local-volumes-test-8279" to be "Unschedulable" May 15 01:52:24.318: INFO: Pod "pod-5e443b34-2e92-4619-9818-fdf83e72b807": Phase="Pending", Reason="", readiness=false. Elapsed: 2.866161ms May 15 01:52:26.322: INFO: Pod "pod-5e443b34-2e92-4619-9818-fdf83e72b807": Phase="Pending", Reason="", readiness=false. Elapsed: 2.006706208s May 15 01:52:26.322: INFO: Pod "pod-5e443b34-2e92-4619-9818-fdf83e72b807" satisfied condition "Unschedulable" [AfterEach] Pod with node different from PV's NodeAffinity /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:356 STEP: Cleaning up PVC and PV May 15 01:52:26.322: INFO: Deleting PersistentVolumeClaim "pvc-j72cf" May 15 01:52:26.326: INFO: Deleting PersistentVolume "local-pvk98lk" STEP: Removing the test directory May 15 01:52:26.330: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-0eb1e386-d168-4ca3-baa7-f6a17212ee9a] Namespace:persistent-local-volumes-test-8279 PodName:hostexec-node1-xb8p5 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:52:26.330: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:52:26.458: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-8279" for this suite. • [SLOW TEST:8.459 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 Pod with node different from PV's NodeAffinity /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:339 should fail scheduling due to different NodeAffinity /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:361 ------------------------------ {"msg":"PASSED [sig-storage] PersistentVolumes-local Pod with node different from PV's NodeAffinity should fail scheduling due to different NodeAffinity","total":-1,"completed":5,"skipped":729,"failed":2,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] One pod requesting one prebound PVC should be able to mount volume and write from pod1","[sig-storage] PersistentVolumes-local [Volume type: blockfswithformat] One pod requesting one prebound PVC should be able to mount volume and write from pod1"]} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:52:19.833: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: blockfswithformat] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes STEP: Creating block device on node "node1" using path "/tmp/local-volume-test-620951f2-90d9-4b3d-98cc-eac93a560d06" May 15 01:52:23.880: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-620951f2-90d9-4b3d-98cc-eac93a560d06 && dd if=/dev/zero of=/tmp/local-volume-test-620951f2-90d9-4b3d-98cc-eac93a560d06/file bs=4096 count=5120 && losetup -f /tmp/local-volume-test-620951f2-90d9-4b3d-98cc-eac93a560d06/file] Namespace:persistent-local-volumes-test-625 PodName:hostexec-node1-tnz85 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:52:23.880: INFO: >>> kubeConfig: /root/.kube/config May 15 01:52:24.019: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c E2E_LOOP_DEV=$(losetup | grep /tmp/local-volume-test-620951f2-90d9-4b3d-98cc-eac93a560d06/file | awk '{ print $1 }') 2>&1 > /dev/null && echo ${E2E_LOOP_DEV}] Namespace:persistent-local-volumes-test-625 PodName:hostexec-node1-tnz85 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:52:24.019: INFO: >>> kubeConfig: /root/.kube/config May 15 01:52:24.146: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkfs -t ext4 /dev/loop2 && mount -t ext4 /dev/loop2 /tmp/local-volume-test-620951f2-90d9-4b3d-98cc-eac93a560d06 && chmod o+rwx /tmp/local-volume-test-620951f2-90d9-4b3d-98cc-eac93a560d06] Namespace:persistent-local-volumes-test-625 PodName:hostexec-node1-tnz85 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:52:24.146: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:52:24.411: INFO: Creating a PV followed by a PVC May 15 01:52:24.417: INFO: Waiting for PV local-pvjrfwl to bind to PVC pvc-nd6xd May 15 01:52:24.417: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-nd6xd] to have phase Bound May 15 01:52:24.420: INFO: PersistentVolumeClaim pvc-nd6xd found but phase is Pending instead of Bound. May 15 01:52:26.425: INFO: PersistentVolumeClaim pvc-nd6xd found but phase is Pending instead of Bound. May 15 01:52:28.429: INFO: PersistentVolumeClaim pvc-nd6xd found and phase=Bound (4.011969273s) May 15 01:52:28.429: INFO: Waiting up to 3m0s for PersistentVolume local-pvjrfwl to have phase Bound May 15 01:52:28.432: INFO: PersistentVolume local-pvjrfwl found and phase=Bound (2.213448ms) [BeforeEach] Set fsGroup for local volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:257 [It] should set different fsGroup for second pod if first pod is deleted /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:282 May 15 01:52:28.436: INFO: Disabled temporarily, reopen after #73168 is fixed [AfterEach] [Volume type: blockfswithformat] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:52:28.437: INFO: Deleting PersistentVolumeClaim "pvc-nd6xd" May 15 01:52:28.443: INFO: Deleting PersistentVolume "local-pvjrfwl" May 15 01:52:28.447: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c umount /tmp/local-volume-test-620951f2-90d9-4b3d-98cc-eac93a560d06] Namespace:persistent-local-volumes-test-625 PodName:hostexec-node1-tnz85 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:52:28.447: INFO: >>> kubeConfig: /root/.kube/config May 15 01:52:28.578: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c E2E_LOOP_DEV=$(losetup | grep /tmp/local-volume-test-620951f2-90d9-4b3d-98cc-eac93a560d06/file | awk '{ print $1 }') 2>&1 > /dev/null && echo ${E2E_LOOP_DEV}] Namespace:persistent-local-volumes-test-625 PodName:hostexec-node1-tnz85 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:52:28.578: INFO: >>> kubeConfig: /root/.kube/config STEP: Tear down block device "/dev/loop2" on node "node1" at path /tmp/local-volume-test-620951f2-90d9-4b3d-98cc-eac93a560d06/file May 15 01:52:28.730: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c losetup -d /dev/loop2] Namespace:persistent-local-volumes-test-625 PodName:hostexec-node1-tnz85 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:52:28.730: INFO: >>> kubeConfig: /root/.kube/config STEP: Removing the test directory /tmp/local-volume-test-620951f2-90d9-4b3d-98cc-eac93a560d06 May 15 01:52:28.849: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-620951f2-90d9-4b3d-98cc-eac93a560d06] Namespace:persistent-local-volumes-test-625 PodName:hostexec-node1-tnz85 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:52:28.849: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:52:28.976: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-625" for this suite. S [SKIPPING] [9.151 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: blockfswithformat] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 Set fsGroup for local volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:256 should set different fsGroup for second pod if first pod is deleted [It] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:282 Disabled temporarily, reopen after #73168 is fixed /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:283 ------------------------------ SSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] Pod Disks /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:52:29.028: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename pod-disks STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] Pod Disks /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/pd.go:75 May 15 01:52:29.048: INFO: Requires at least 2 nodes (not -1) [AfterEach] [sig-storage] Pod Disks /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:52:29.050: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "pod-disks-6311" for this suite. S [SKIPPING] in Spec Setup (BeforeEach) [0.030 seconds] [sig-storage] Pod Disks /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 should be able to delete a non-existent PD without error [BeforeEach] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/pd.go:448 Requires at least 2 nodes (not -1) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/pd.go:76 ------------------------------ SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:50:29.230: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: dir-bindmounted] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes May 15 01:50:33.282: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir /tmp/local-volume-test-54fa3caa-17a5-476f-b4e1-d15ddbb03d15 && mount --bind /tmp/local-volume-test-54fa3caa-17a5-476f-b4e1-d15ddbb03d15 /tmp/local-volume-test-54fa3caa-17a5-476f-b4e1-d15ddbb03d15] Namespace:persistent-local-volumes-test-234 PodName:hostexec-node1-2l9b5 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:50:33.282: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:50:33.412: INFO: Creating a PV followed by a PVC May 15 01:50:33.420: INFO: Waiting for PV local-pvv6kk2 to bind to PVC pvc-c8ttk May 15 01:50:33.420: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-c8ttk] to have phase Bound May 15 01:50:33.422: INFO: PersistentVolumeClaim pvc-c8ttk found but phase is Pending instead of Bound. May 15 01:50:35.424: INFO: PersistentVolumeClaim pvc-c8ttk found and phase=Bound (2.004684453s) May 15 01:50:35.424: INFO: Waiting up to 3m0s for PersistentVolume local-pvv6kk2 to have phase Bound May 15 01:50:35.427: INFO: PersistentVolume local-pvv6kk2 found and phase=Bound (2.709177ms) [It] should be able to write from pod1 and read from pod2 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:251 STEP: Creating pod1 STEP: Creating a pod May 15 01:52:35.450: FAIL: Unexpected error: <*errors.errorString | 0xc00315e2b0>: { s: "pod \"pod-58f129b2-416d-48d8-93b6-2628e4cc65d5\" is not Running: timed out waiting for the condition", } pod "pod-58f129b2-416d-48d8-93b6-2628e4cc65d5" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage.twoPodsReadWriteSerialTest(0xc002ea9760, 0xc000d62c80, 0xc00542bdd0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:764 +0xaf k8s.io/kubernetes/test/e2e/storage.glob..func20.2.5.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:252 +0x45 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000703800) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc000703800) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc000703800, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [Volume type: dir-bindmounted] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:52:35.452: INFO: Deleting PersistentVolumeClaim "pvc-c8ttk" May 15 01:52:35.456: INFO: Deleting PersistentVolume "local-pvv6kk2" STEP: Removing the test directory May 15 01:52:35.460: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c umount /tmp/local-volume-test-54fa3caa-17a5-476f-b4e1-d15ddbb03d15 && rm -r /tmp/local-volume-test-54fa3caa-17a5-476f-b4e1-d15ddbb03d15] Namespace:persistent-local-volumes-test-234 PodName:hostexec-node1-2l9b5 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:52:35.460: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "persistent-local-volumes-test-234". STEP: Found 14 events. May 15 01:52:35.624: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for hostexec-node1-2l9b5: { } Scheduled: Successfully assigned persistent-local-volumes-test-234/hostexec-node1-2l9b5 to node1 May 15 01:52:35.624: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-58f129b2-416d-48d8-93b6-2628e4cc65d5: { } Scheduled: Successfully assigned persistent-local-volumes-test-234/pod-58f129b2-416d-48d8-93b6-2628e4cc65d5 to node1 May 15 01:52:35.624: INFO: At 2021-05-15 01:50:29 +0000 UTC - event for hostexec-node1-2l9b5: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 15 01:52:35.624: INFO: At 2021-05-15 01:50:31 +0000 UTC - event for hostexec-node1-2l9b5: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 1.115250335s May 15 01:52:35.624: INFO: At 2021-05-15 01:50:31 +0000 UTC - event for hostexec-node1-2l9b5: {kubelet node1} Created: Created container agnhost-container May 15 01:52:35.624: INFO: At 2021-05-15 01:50:31 +0000 UTC - event for hostexec-node1-2l9b5: {kubelet node1} Started: Started container agnhost-container May 15 01:52:35.624: INFO: At 2021-05-15 01:50:37 +0000 UTC - event for pod-58f129b2-416d-48d8-93b6-2628e4cc65d5: {multus } AddedInterface: Add eth0 [10.244.3.177/24] May 15 01:52:35.624: INFO: At 2021-05-15 01:50:37 +0000 UTC - event for pod-58f129b2-416d-48d8-93b6-2628e4cc65d5: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:52:35.624: INFO: At 2021-05-15 01:50:39 +0000 UTC - event for pod-58f129b2-416d-48d8-93b6-2628e4cc65d5: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:52:35.624: INFO: At 2021-05-15 01:50:39 +0000 UTC - event for pod-58f129b2-416d-48d8-93b6-2628e4cc65d5: {kubelet node1} Failed: Error: ErrImagePull May 15 01:52:35.624: INFO: At 2021-05-15 01:50:39 +0000 UTC - event for pod-58f129b2-416d-48d8-93b6-2628e4cc65d5: {kubelet node1} SandboxChanged: Pod sandbox changed, it will be killed and re-created. May 15 01:52:35.624: INFO: At 2021-05-15 01:50:41 +0000 UTC - event for pod-58f129b2-416d-48d8-93b6-2628e4cc65d5: {multus } AddedInterface: Add eth0 [10.244.3.178/24] May 15 01:52:35.624: INFO: At 2021-05-15 01:50:41 +0000 UTC - event for pod-58f129b2-416d-48d8-93b6-2628e4cc65d5: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 15 01:52:35.624: INFO: At 2021-05-15 01:50:41 +0000 UTC - event for pod-58f129b2-416d-48d8-93b6-2628e4cc65d5: {kubelet node1} Failed: Error: ImagePullBackOff May 15 01:52:35.626: INFO: POD NODE PHASE GRACE CONDITIONS May 15 01:52:35.626: INFO: hostexec-node1-2l9b5 node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:50:29 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:50:32 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:50:32 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:50:29 +0000 UTC }] May 15 01:52:35.626: INFO: pod-58f129b2-416d-48d8-93b6-2628e4cc65d5 node1 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:50:35 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:50:35 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:50:35 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:50:35 +0000 UTC }] May 15 01:52:35.626: INFO: May 15 01:52:35.631: INFO: Logging node info for node master1 May 15 01:52:35.633: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 160288 0 2021-05-14 19:56:35 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:29 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:29 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:29 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:52:29 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[@ :],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:52:35.634: INFO: Logging kubelet events for node master1 May 15 01:52:35.639: INFO: Logging pods the kubelet thinks is on node master1 May 15 01:52:35.648: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.648: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:52:35.648: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:52:35.648: INFO: Init container install-cni ready: true, restart count 0 May 15 01:52:35.648: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:52:35.648: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:52:35.648: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:52:35.648: INFO: Container node-exporter ready: true, restart count 0 May 15 01:52:35.648: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.648: INFO: Container kube-scheduler ready: true, restart count 0 May 15 01:52:35.648: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.649: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:52:35.649: INFO: kubernetes-metrics-scraper-678c97765c-fswrn started at 2021-05-15 00:18:49 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.649: INFO: Container kubernetes-metrics-scraper ready: true, restart count 0 May 15 01:52:35.649: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.649: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:52:35.649: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.649: INFO: Container kube-multus ready: true, restart count 1 May 15 01:52:35.649: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 15 01:52:35.649: INFO: Container docker-registry ready: true, restart count 0 May 15 01:52:35.649: INFO: Container nginx ready: true, restart count 0 May 15 01:52:35.649: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.649: INFO: Container nfd-controller ready: true, restart count 0 W0515 01:52:35.662265 34 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:52:35.689: INFO: Latency metrics for node master1 May 15 01:52:35.689: INFO: Logging node info for node master2 May 15 01:52:35.691: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 160255 0 2021-05-14 19:57:04 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:26 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:26 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:26 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:52:26 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:52:35.692: INFO: Logging kubelet events for node master2 May 15 01:52:35.696: INFO: Logging pods the kubelet thinks is on node master2 May 15 01:52:35.705: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.705: INFO: Container autoscaler ready: true, restart count 2 May 15 01:52:35.705: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:52:35.705: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:52:35.705: INFO: Container node-exporter ready: true, restart count 0 May 15 01:52:35.705: INFO: prometheus-operator-5bb8cb9d8f-fqb87 started at 2021-05-15 00:18:49 +0000 UTC (0+2 container statuses recorded) May 15 01:52:35.705: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:52:35.705: INFO: Container prometheus-operator ready: true, restart count 0 May 15 01:52:35.705: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.705: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:52:35.705: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.705: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:52:35.705: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.705: INFO: Container kube-scheduler ready: true, restart count 2 May 15 01:52:35.705: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:52:35.705: INFO: Init container install-cni ready: true, restart count 0 May 15 01:52:35.705: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:52:35.705: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.705: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:52:35.705: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.706: INFO: Container kube-multus ready: true, restart count 1 May 15 01:52:35.706: INFO: coredns-7677f9bb54-96w24 started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.706: INFO: Container coredns ready: true, restart count 0 W0515 01:52:35.718935 34 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:52:35.751: INFO: Latency metrics for node master2 May 15 01:52:35.751: INFO: Logging node info for node master3 May 15 01:52:35.753: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 160436 0 2021-05-14 19:57:14 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234767872 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324603904 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:35 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:35 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:35 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:52:35 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:52:35.754: INFO: Logging kubelet events for node master3 May 15 01:52:35.758: INFO: Logging pods the kubelet thinks is on node master3 May 15 01:52:35.765: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.765: INFO: Container kube-controller-manager ready: true, restart count 3 May 15 01:52:35.765: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.765: INFO: Container kube-scheduler ready: true, restart count 3 May 15 01:52:35.765: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.765: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:52:35.765: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.765: INFO: Container kube-proxy ready: true, restart count 1 May 15 01:52:35.765: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:52:35.765: INFO: Init container install-cni ready: true, restart count 0 May 15 01:52:35.765: INFO: Container kube-flannel ready: true, restart count 2 May 15 01:52:35.765: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.765: INFO: Container kube-multus ready: true, restart count 1 May 15 01:52:35.765: INFO: coredns-7677f9bb54-rpj8c started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.765: INFO: Container coredns ready: true, restart count 0 May 15 01:52:35.765: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:52:35.765: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:52:35.765: INFO: Container node-exporter ready: true, restart count 0 W0515 01:52:35.779259 34 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:52:35.808: INFO: Latency metrics for node master3 May 15 01:52:35.808: INFO: Logging node info for node node1 May 15 01:52:35.811: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 160403 0 2021-05-14 19:58:21 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1602":"csi-mock-csi-mock-volumes-1602","csi-mock-csi-mock-volumes-253":"csi-mock-csi-mock-volumes-253","csi-mock-csi-mock-volumes-2993":"csi-mock-csi-mock-volumes-2993","csi-mock-csi-mock-volumes-7734":"csi-mock-csi-mock-volumes-7734","csi-mock-csi-mock-volumes-895":"csi-mock-csi-mock-volumes-895","csi-mock-csi-mock-volumes-9728":"csi-mock-csi-mock-volumes-9728","csi-mock-csi-mock-volumes-9809":"csi-mock-csi-mock-volumes-9809"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:00 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:50:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:51:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:26 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:26 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:26 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:52:26 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[@ :],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:52:35.812: INFO: Logging kubelet events for node node1 May 15 01:52:35.817: INFO: Logging pods the kubelet thinks is on node node1 May 15 01:52:35.838: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.838: INFO: Container kube-multus ready: true, restart count 1 May 15 01:52:35.838: INFO: prometheus-k8s-0 started at 2021-05-15 00:19:01 +0000 UTC (0+5 container statuses recorded) May 15 01:52:35.838: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 15 01:52:35.838: INFO: Container grafana ready: true, restart count 0 May 15 01:52:35.838: INFO: Container prometheus ready: true, restart count 26 May 15 01:52:35.838: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 15 01:52:35.838: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 15 01:52:35.838: INFO: hostexec-node1-ktptr started at 2021-05-15 01:50:37 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.838: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:52:35.838: INFO: node-exporter-flvqz started at 2021-05-15 00:18:55 +0000 UTC (0+2 container statuses recorded) May 15 01:52:35.838: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:52:35.838: INFO: Container node-exporter ready: true, restart count 0 May 15 01:52:35.838: INFO: pod-58f129b2-416d-48d8-93b6-2628e4cc65d5 started at 2021-05-15 01:50:35 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.838: INFO: Container write-pod ready: false, restart count 0 May 15 01:52:35.838: INFO: pod-bfba8786-cc42-4bcf-babe-8aa8c1a6f88f started at 2021-05-15 01:51:46 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.838: INFO: Container write-pod ready: false, restart count 0 May 15 01:52:35.838: INFO: pod-6fec197a-289d-4046-b7a7-00d385f2dc10 started at 2021-05-15 01:50:35 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.838: INFO: Container write-pod ready: false, restart count 0 May 15 01:52:35.838: INFO: cmk-4s6dm started at 2021-05-15 00:18:54 +0000 UTC (0+2 container statuses recorded) May 15 01:52:35.838: INFO: Container nodereport ready: true, restart count 0 May 15 01:52:35.838: INFO: Container reconcile ready: true, restart count 0 May 15 01:52:35.838: INFO: pod-9566f4e9-ff2d-489c-94bf-709e28a35a7d started at 2021-05-15 01:50:43 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.838: INFO: Container write-pod ready: false, restart count 0 May 15 01:52:35.838: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.838: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:52:35.838: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4vxcc started at 2021-05-15 00:19:00 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.838: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:52:35.838: INFO: hostexec-node1-vlzvd started at 2021-05-15 01:51:39 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.838: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:52:35.838: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.838: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:52:35.838: INFO: hostexec-node1-c8b5r started at 2021-05-15 01:52:05 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.838: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:52:35.838: INFO: hostexec-node1-2l9b5 started at 2021-05-15 01:50:29 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.839: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:52:35.839: INFO: pod-83dd1c20-b22c-4938-b3e4-dcd2105d871b started at 2021-05-15 01:52:13 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.839: INFO: Container write-pod ready: false, restart count 0 May 15 01:52:35.839: INFO: hostexec-node1-dwvtr started at 2021-05-15 01:50:31 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.839: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:52:35.839: INFO: hostexec-node1-zbxmq started at 2021-05-15 01:50:51 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.839: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:52:35.839: INFO: hostexec-node1-qjs8r started at 2021-05-15 01:52:26 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.839: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:52:35.839: INFO: csi-mockplugin-0 started at 2021-05-15 01:52:29 +0000 UTC (0+3 container statuses recorded) May 15 01:52:35.839: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:52:35.839: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:52:35.839: INFO: Container mock ready: true, restart count 0 May 15 01:52:35.839: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:52:35.839: INFO: Init container install-cni ready: true, restart count 0 May 15 01:52:35.839: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:52:35.839: INFO: node-feature-discovery-worker-bw8zg started at 2021-05-15 00:18:56 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.839: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:52:35.839: INFO: collectd-mrzps started at 2021-05-15 00:19:22 +0000 UTC (0+3 container statuses recorded) May 15 01:52:35.839: INFO: Container collectd ready: true, restart count 0 May 15 01:52:35.839: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:52:35.839: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:52:35.839: INFO: pod-11760848-bbb5-4ae1-af82-a7a2a4af6a9c started at 2021-05-15 01:50:59 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.839: INFO: Container write-pod ready: false, restart count 0 W0515 01:52:35.852354 34 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:52:35.905: INFO: Latency metrics for node node1 May 15 01:52:35.905: INFO: Logging node info for node node2 May 15 01:52:35.908: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 160359 0 2021-05-14 19:58:22 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1663":"csi-mock-csi-mock-volumes-1663","csi-mock-csi-mock-volumes-3052":"csi-mock-csi-mock-volumes-3052","csi-mock-csi-mock-volumes-5200":"csi-mock-csi-mock-volumes-5200","csi-mock-csi-mock-volumes-5678":"csi-mock-csi-mock-volumes-5678","csi-mock-csi-mock-volumes-8760":"csi-mock-csi-mock-volumes-8760","csi-mock-csi-mock-volumes-9624":"csi-mock-csi-mock-volumes-9624"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:46 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:52:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:52:21 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:io.kubernetes.storage.mock/node":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:31 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:31 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:31 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:52:31 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:52:35.909: INFO: Logging kubelet events for node node2 May 15 01:52:35.914: INFO: Logging pods the kubelet thinks is on node node2 May 15 01:52:35.926: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.926: INFO: Container cmk-webhook ready: true, restart count 0 May 15 01:52:35.926: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 15 01:52:35.926: INFO: Container discover ready: false, restart count 0 May 15 01:52:35.926: INFO: Container init ready: false, restart count 0 May 15 01:52:35.926: INFO: Container install ready: false, restart count 0 May 15 01:52:35.926: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.926: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:52:35.926: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.926: INFO: Container kube-multus ready: true, restart count 1 May 15 01:52:35.926: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.926: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 15 01:52:35.926: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.926: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:52:35.926: INFO: csi-mockplugin-0 started at 2021-05-15 01:52:00 +0000 UTC (0+3 container statuses recorded) May 15 01:52:35.926: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:52:35.926: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:52:35.926: INFO: Container mock ready: true, restart count 0 May 15 01:52:35.926: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:52:35.926: INFO: Init container install-cni ready: true, restart count 1 May 15 01:52:35.926: INFO: Container kube-flannel ready: true, restart count 4 May 15 01:52:35.926: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:52:35.926: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:52:35.926: INFO: Container node-exporter ready: true, restart count 0 May 15 01:52:35.926: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 15 01:52:35.926: INFO: Container tas-controller ready: true, restart count 0 May 15 01:52:35.926: INFO: Container tas-extender ready: true, restart count 0 May 15 01:52:35.926: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 15 01:52:35.926: INFO: Container collectd ready: true, restart count 0 May 15 01:52:35.926: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:52:35.926: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:52:35.926: INFO: csi-mockplugin-attacher-0 started at 2021-05-15 01:52:00 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.926: INFO: Container csi-attacher ready: true, restart count 0 May 15 01:52:35.926: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.926: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:52:35.926: INFO: pod-subpath-test-configmap-lp9m started at 2021-05-15 01:50:05 +0000 UTC (1+2 container statuses recorded) May 15 01:52:35.926: INFO: Init container init-volume-configmap-lp9m ready: false, restart count 0 May 15 01:52:35.926: INFO: Container test-container-subpath-configmap-lp9m ready: false, restart count 0 May 15 01:52:35.926: INFO: Container test-container-volume-configmap-lp9m ready: false, restart count 0 May 15 01:52:35.926: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.926: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:52:35.926: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 15 01:52:35.926: INFO: Container nodereport ready: true, restart count 0 May 15 01:52:35.926: INFO: Container reconcile ready: true, restart count 0 W0515 01:52:35.939592 34 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:52:35.983: INFO: Latency metrics for node node2 May 15 01:52:35.983: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-234" for this suite. • Failure [126.761 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: dir-bindmounted] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 Two pods mounting a local volume one after the other /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:250 should be able to write from pod1 and read from pod2 [It] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:251 May 15 01:52:35.450: Unexpected error: <*errors.errorString | 0xc00315e2b0>: { s: "pod \"pod-58f129b2-416d-48d8-93b6-2628e4cc65d5\" is not Running: timed out waiting for the condition", } pod "pod-58f129b2-416d-48d8-93b6-2628e4cc65d5" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:764 ------------------------------ {"msg":"FAILED [sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2","total":-1,"completed":0,"skipped":78,"failed":4,"failures":["[sig-storage] PersistentVolumes-local [Volume type: blockfswithoutformat] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir-link-bindmounted] One pod requesting one prebound PVC should be able to mount volume and read from pod1","[sig-storage] PersistentVolumes-local [Volume type: blockfswithoutformat] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2"]} SSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:50:31.360: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: dir] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes May 15 01:50:33.407: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-c7b613e9-2da0-439d-88d4-6f10735b9177] Namespace:persistent-local-volumes-test-9996 PodName:hostexec-node1-dwvtr ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:50:33.407: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:50:33.528: INFO: Creating a PV followed by a PVC May 15 01:50:33.535: INFO: Waiting for PV local-pvq6nz5 to bind to PVC pvc-hnc4t May 15 01:50:33.535: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-hnc4t] to have phase Bound May 15 01:50:33.538: INFO: PersistentVolumeClaim pvc-hnc4t found but phase is Pending instead of Bound. May 15 01:50:35.542: INFO: PersistentVolumeClaim pvc-hnc4t found and phase=Bound (2.006474707s) May 15 01:50:35.542: INFO: Waiting up to 3m0s for PersistentVolume local-pvq6nz5 to have phase Bound May 15 01:50:35.544: INFO: PersistentVolume local-pvq6nz5 found and phase=Bound (1.68533ms) [BeforeEach] One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:211 STEP: Creating pod1 STEP: Creating a pod May 15 01:52:35.568: FAIL: Unexpected error: <*errors.errorString | 0xc004f267b0>: { s: "pod \"pod-6fec197a-289d-4046-b7a7-00d385f2dc10\" is not Running: timed out waiting for the condition", } pod "pod-6fec197a-289d-4046-b7a7-00d385f2dc10" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage.glob..func20.2.3.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 +0x119 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc002904000) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc002904000) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc002904000, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:223 STEP: Deleting pod1 STEP: Deleting pod pod-6fec197a-289d-4046-b7a7-00d385f2dc10 in namespace persistent-local-volumes-test-9996 [AfterEach] [Volume type: dir] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:52:35.574: INFO: Deleting PersistentVolumeClaim "pvc-hnc4t" May 15 01:52:35.578: INFO: Deleting PersistentVolume "local-pvq6nz5" STEP: Removing the test directory May 15 01:52:35.583: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-c7b613e9-2da0-439d-88d4-6f10735b9177] Namespace:persistent-local-volumes-test-9996 PodName:hostexec-node1-dwvtr ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:52:35.583: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "persistent-local-volumes-test-9996". STEP: Found 12 events. May 15 01:52:35.710: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for hostexec-node1-dwvtr: { } Scheduled: Successfully assigned persistent-local-volumes-test-9996/hostexec-node1-dwvtr to node1 May 15 01:52:35.710: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-6fec197a-289d-4046-b7a7-00d385f2dc10: { } Scheduled: Successfully assigned persistent-local-volumes-test-9996/pod-6fec197a-289d-4046-b7a7-00d385f2dc10 to node1 May 15 01:52:35.710: INFO: At 2021-05-15 01:50:32 +0000 UTC - event for hostexec-node1-dwvtr: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 15 01:52:35.710: INFO: At 2021-05-15 01:50:32 +0000 UTC - event for hostexec-node1-dwvtr: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 478.457131ms May 15 01:52:35.710: INFO: At 2021-05-15 01:50:32 +0000 UTC - event for hostexec-node1-dwvtr: {kubelet node1} Created: Created container agnhost-container May 15 01:52:35.710: INFO: At 2021-05-15 01:50:32 +0000 UTC - event for hostexec-node1-dwvtr: {kubelet node1} Started: Started container agnhost-container May 15 01:52:35.710: INFO: At 2021-05-15 01:50:37 +0000 UTC - event for pod-6fec197a-289d-4046-b7a7-00d385f2dc10: {multus } AddedInterface: Add eth0 [10.244.3.176/24] May 15 01:52:35.710: INFO: At 2021-05-15 01:50:37 +0000 UTC - event for pod-6fec197a-289d-4046-b7a7-00d385f2dc10: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:52:35.710: INFO: At 2021-05-15 01:50:38 +0000 UTC - event for pod-6fec197a-289d-4046-b7a7-00d385f2dc10: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:52:35.710: INFO: At 2021-05-15 01:50:38 +0000 UTC - event for pod-6fec197a-289d-4046-b7a7-00d385f2dc10: {kubelet node1} Failed: Error: ErrImagePull May 15 01:52:35.710: INFO: At 2021-05-15 01:50:38 +0000 UTC - event for pod-6fec197a-289d-4046-b7a7-00d385f2dc10: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 15 01:52:35.710: INFO: At 2021-05-15 01:50:38 +0000 UTC - event for pod-6fec197a-289d-4046-b7a7-00d385f2dc10: {kubelet node1} Failed: Error: ImagePullBackOff May 15 01:52:35.713: INFO: POD NODE PHASE GRACE CONDITIONS May 15 01:52:35.713: INFO: hostexec-node1-dwvtr node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:50:31 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:50:33 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:50:33 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:50:31 +0000 UTC }] May 15 01:52:35.713: INFO: pod-6fec197a-289d-4046-b7a7-00d385f2dc10 node1 Pending 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:50:35 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:50:35 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:50:35 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:50:35 +0000 UTC }] May 15 01:52:35.713: INFO: May 15 01:52:35.717: INFO: Logging node info for node master1 May 15 01:52:35.719: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 160288 0 2021-05-14 19:56:35 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:29 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:29 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:29 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:52:29 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[@ :],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:52:35.719: INFO: Logging kubelet events for node master1 May 15 01:52:35.725: INFO: Logging pods the kubelet thinks is on node master1 May 15 01:52:35.733: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:52:35.733: INFO: Init container install-cni ready: true, restart count 0 May 15 01:52:35.733: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:52:35.733: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:52:35.733: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:52:35.733: INFO: Container node-exporter ready: true, restart count 0 May 15 01:52:35.733: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.733: INFO: Container kube-scheduler ready: true, restart count 0 May 15 01:52:35.733: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.733: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:52:35.733: INFO: kubernetes-metrics-scraper-678c97765c-fswrn started at 2021-05-15 00:18:49 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.733: INFO: Container kubernetes-metrics-scraper ready: true, restart count 0 May 15 01:52:35.733: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.733: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:52:35.733: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.733: INFO: Container kube-multus ready: true, restart count 1 May 15 01:52:35.733: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 15 01:52:35.733: INFO: Container docker-registry ready: true, restart count 0 May 15 01:52:35.733: INFO: Container nginx ready: true, restart count 0 May 15 01:52:35.733: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.733: INFO: Container nfd-controller ready: true, restart count 0 May 15 01:52:35.733: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.733: INFO: Container kube-apiserver ready: true, restart count 0 W0515 01:52:35.746131 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:52:35.773: INFO: Latency metrics for node master1 May 15 01:52:35.773: INFO: Logging node info for node master2 May 15 01:52:35.775: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 160255 0 2021-05-14 19:57:04 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:26 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:26 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:26 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:52:26 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:52:35.775: INFO: Logging kubelet events for node master2 May 15 01:52:35.779: INFO: Logging pods the kubelet thinks is on node master2 May 15 01:52:35.788: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.788: INFO: Container kube-scheduler ready: true, restart count 2 May 15 01:52:35.788: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:52:35.788: INFO: Init container install-cni ready: true, restart count 0 May 15 01:52:35.788: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:52:35.788: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.788: INFO: Container autoscaler ready: true, restart count 2 May 15 01:52:35.788: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:52:35.788: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:52:35.788: INFO: Container node-exporter ready: true, restart count 0 May 15 01:52:35.788: INFO: prometheus-operator-5bb8cb9d8f-fqb87 started at 2021-05-15 00:18:49 +0000 UTC (0+2 container statuses recorded) May 15 01:52:35.788: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:52:35.788: INFO: Container prometheus-operator ready: true, restart count 0 May 15 01:52:35.788: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.788: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:52:35.788: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.788: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:52:35.788: INFO: coredns-7677f9bb54-96w24 started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.788: INFO: Container coredns ready: true, restart count 0 May 15 01:52:35.788: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.788: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:52:35.788: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.788: INFO: Container kube-multus ready: true, restart count 1 W0515 01:52:35.800835 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:52:35.825: INFO: Latency metrics for node master2 May 15 01:52:35.825: INFO: Logging node info for node master3 May 15 01:52:35.828: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 160436 0 2021-05-14 19:57:14 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234767872 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324603904 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:35 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:35 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:35 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:52:35 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:52:35.828: INFO: Logging kubelet events for node master3 May 15 01:52:35.832: INFO: Logging pods the kubelet thinks is on node master3 May 15 01:52:35.840: INFO: coredns-7677f9bb54-rpj8c started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.840: INFO: Container coredns ready: true, restart count 0 May 15 01:52:35.840: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:52:35.840: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:52:35.840: INFO: Container node-exporter ready: true, restart count 0 May 15 01:52:35.840: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.840: INFO: Container kube-controller-manager ready: true, restart count 3 May 15 01:52:35.840: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.840: INFO: Container kube-scheduler ready: true, restart count 3 May 15 01:52:35.840: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.840: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:52:35.840: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.840: INFO: Container kube-proxy ready: true, restart count 1 May 15 01:52:35.840: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:52:35.840: INFO: Init container install-cni ready: true, restart count 0 May 15 01:52:35.840: INFO: Container kube-flannel ready: true, restart count 2 May 15 01:52:35.840: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.840: INFO: Container kube-multus ready: true, restart count 1 W0515 01:52:35.854443 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:52:35.879: INFO: Latency metrics for node master3 May 15 01:52:35.879: INFO: Logging node info for node node1 May 15 01:52:35.882: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 160403 0 2021-05-14 19:58:21 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1602":"csi-mock-csi-mock-volumes-1602","csi-mock-csi-mock-volumes-253":"csi-mock-csi-mock-volumes-253","csi-mock-csi-mock-volumes-2993":"csi-mock-csi-mock-volumes-2993","csi-mock-csi-mock-volumes-7734":"csi-mock-csi-mock-volumes-7734","csi-mock-csi-mock-volumes-895":"csi-mock-csi-mock-volumes-895","csi-mock-csi-mock-volumes-9728":"csi-mock-csi-mock-volumes-9728","csi-mock-csi-mock-volumes-9809":"csi-mock-csi-mock-volumes-9809"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:00 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:50:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:51:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:26 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:26 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:26 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:52:26 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[@ :],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:52:35.883: INFO: Logging kubelet events for node node1 May 15 01:52:35.886: INFO: Logging pods the kubelet thinks is on node node1 May 15 01:52:35.902: INFO: hostexec-node1-dwvtr started at 2021-05-15 01:50:31 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.902: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:52:35.902: INFO: hostexec-node1-zbxmq started at 2021-05-15 01:50:51 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.902: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:52:35.902: INFO: pod-11760848-bbb5-4ae1-af82-a7a2a4af6a9c started at 2021-05-15 01:50:59 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.902: INFO: Container write-pod ready: false, restart count 0 May 15 01:52:35.902: INFO: hostexec-node1-qjs8r started at 2021-05-15 01:52:26 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.902: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:52:35.902: INFO: csi-mockplugin-0 started at 2021-05-15 01:52:29 +0000 UTC (0+3 container statuses recorded) May 15 01:52:35.902: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:52:35.902: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:52:35.902: INFO: Container mock ready: true, restart count 0 May 15 01:52:35.902: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:52:35.902: INFO: Init container install-cni ready: true, restart count 0 May 15 01:52:35.902: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:52:35.902: INFO: node-feature-discovery-worker-bw8zg started at 2021-05-15 00:18:56 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.902: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:52:35.902: INFO: collectd-mrzps started at 2021-05-15 00:19:22 +0000 UTC (0+3 container statuses recorded) May 15 01:52:35.902: INFO: Container collectd ready: true, restart count 0 May 15 01:52:35.902: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:52:35.902: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:52:35.902: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.902: INFO: Container kube-multus ready: true, restart count 1 May 15 01:52:35.902: INFO: prometheus-k8s-0 started at 2021-05-15 00:19:01 +0000 UTC (0+5 container statuses recorded) May 15 01:52:35.902: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 15 01:52:35.902: INFO: Container grafana ready: true, restart count 0 May 15 01:52:35.902: INFO: Container prometheus ready: true, restart count 26 May 15 01:52:35.902: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 15 01:52:35.902: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 15 01:52:35.902: INFO: pod-bfba8786-cc42-4bcf-babe-8aa8c1a6f88f started at 2021-05-15 01:51:46 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.902: INFO: Container write-pod ready: false, restart count 0 May 15 01:52:35.902: INFO: pod-6fec197a-289d-4046-b7a7-00d385f2dc10 started at 2021-05-15 01:50:35 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.902: INFO: Container write-pod ready: false, restart count 0 May 15 01:52:35.902: INFO: hostexec-node1-ktptr started at 2021-05-15 01:50:37 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.902: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:52:35.902: INFO: node-exporter-flvqz started at 2021-05-15 00:18:55 +0000 UTC (0+2 container statuses recorded) May 15 01:52:35.902: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:52:35.902: INFO: Container node-exporter ready: true, restart count 0 May 15 01:52:35.902: INFO: pod-58f129b2-416d-48d8-93b6-2628e4cc65d5 started at 2021-05-15 01:50:35 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.902: INFO: Container write-pod ready: false, restart count 0 May 15 01:52:35.902: INFO: pod-9566f4e9-ff2d-489c-94bf-709e28a35a7d started at 2021-05-15 01:50:43 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.902: INFO: Container write-pod ready: false, restart count 0 May 15 01:52:35.902: INFO: cmk-4s6dm started at 2021-05-15 00:18:54 +0000 UTC (0+2 container statuses recorded) May 15 01:52:35.902: INFO: Container nodereport ready: true, restart count 0 May 15 01:52:35.902: INFO: Container reconcile ready: true, restart count 0 May 15 01:52:35.902: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.902: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:52:35.902: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4vxcc started at 2021-05-15 00:19:00 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.902: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:52:35.902: INFO: hostexec-node1-vlzvd started at 2021-05-15 01:51:39 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.902: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:52:35.902: INFO: pod-83dd1c20-b22c-4938-b3e4-dcd2105d871b started at 2021-05-15 01:52:13 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.903: INFO: Container write-pod ready: false, restart count 0 May 15 01:52:35.903: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.903: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:52:35.903: INFO: hostexec-node1-c8b5r started at 2021-05-15 01:52:05 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.903: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:52:35.903: INFO: hostexec-node1-2l9b5 started at 2021-05-15 01:50:29 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.903: INFO: Container agnhost-container ready: true, restart count 0 W0515 01:52:35.913389 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:52:35.978: INFO: Latency metrics for node node1 May 15 01:52:35.978: INFO: Logging node info for node node2 May 15 01:52:35.981: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 160359 0 2021-05-14 19:58:22 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1663":"csi-mock-csi-mock-volumes-1663","csi-mock-csi-mock-volumes-3052":"csi-mock-csi-mock-volumes-3052","csi-mock-csi-mock-volumes-5200":"csi-mock-csi-mock-volumes-5200","csi-mock-csi-mock-volumes-5678":"csi-mock-csi-mock-volumes-5678","csi-mock-csi-mock-volumes-8760":"csi-mock-csi-mock-volumes-8760","csi-mock-csi-mock-volumes-9624":"csi-mock-csi-mock-volumes-9624"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:46 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:52:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:52:21 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:io.kubernetes.storage.mock/node":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:31 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:31 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:31 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:52:31 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:52:35.982: INFO: Logging kubelet events for node node2 May 15 01:52:35.986: INFO: Logging pods the kubelet thinks is on node node2 May 15 01:52:35.998: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.998: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:52:35.998: INFO: csi-mockplugin-0 started at 2021-05-15 01:52:00 +0000 UTC (0+3 container statuses recorded) May 15 01:52:35.998: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:52:35.998: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:52:35.998: INFO: Container mock ready: true, restart count 0 May 15 01:52:35.998: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:52:35.998: INFO: Init container install-cni ready: true, restart count 1 May 15 01:52:35.998: INFO: Container kube-flannel ready: true, restart count 4 May 15 01:52:35.998: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:52:35.998: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:52:35.998: INFO: Container node-exporter ready: true, restart count 0 May 15 01:52:35.998: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 15 01:52:35.998: INFO: Container tas-controller ready: true, restart count 0 May 15 01:52:35.998: INFO: Container tas-extender ready: true, restart count 0 May 15 01:52:35.998: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 15 01:52:35.998: INFO: Container collectd ready: true, restart count 0 May 15 01:52:35.998: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:52:35.998: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:52:35.998: INFO: csi-mockplugin-attacher-0 started at 2021-05-15 01:52:00 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.998: INFO: Container csi-attacher ready: true, restart count 0 May 15 01:52:35.998: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.998: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:52:35.998: INFO: pod-subpath-test-configmap-lp9m started at 2021-05-15 01:50:05 +0000 UTC (1+2 container statuses recorded) May 15 01:52:35.998: INFO: Init container init-volume-configmap-lp9m ready: false, restart count 0 May 15 01:52:35.998: INFO: Container test-container-subpath-configmap-lp9m ready: false, restart count 0 May 15 01:52:35.998: INFO: Container test-container-volume-configmap-lp9m ready: false, restart count 0 May 15 01:52:35.998: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.998: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:52:35.998: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 15 01:52:35.998: INFO: Container nodereport ready: true, restart count 0 May 15 01:52:35.998: INFO: Container reconcile ready: true, restart count 0 May 15 01:52:35.998: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.998: INFO: Container cmk-webhook ready: true, restart count 0 May 15 01:52:35.998: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 15 01:52:35.998: INFO: Container discover ready: false, restart count 0 May 15 01:52:35.998: INFO: Container init ready: false, restart count 0 May 15 01:52:35.998: INFO: Container install ready: false, restart count 0 May 15 01:52:35.999: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.999: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:52:35.999: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.999: INFO: Container kube-multus ready: true, restart count 1 May 15 01:52:35.999: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 15 01:52:35.999: INFO: Container kubernetes-dashboard ready: true, restart count 2 W0515 01:52:36.011432 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:52:36.045: INFO: Latency metrics for node node2 May 15 01:52:36.045: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-9996" for this suite. • Failure in Spec Setup (BeforeEach) [124.695 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: dir] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:205 should be able to mount volume and read from pod1 [BeforeEach] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:228 May 15 01:52:35.568: Unexpected error: <*errors.errorString | 0xc004f267b0>: { s: "pod \"pod-6fec197a-289d-4046-b7a7-00d385f2dc10\" is not Running: timed out waiting for the condition", } pod "pod-6fec197a-289d-4046-b7a7-00d385f2dc10" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 ------------------------------ {"msg":"FAILED [sig-storage] PersistentVolumes-local [Volume type: dir] One pod requesting one prebound PVC should be able to mount volume and read from pod1","total":-1,"completed":4,"skipped":239,"failed":3,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir-link] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir] One pod requesting one prebound PVC should be able to mount volume and read from pod1"]} SSS ------------------------------ [BeforeEach] [sig-storage] ConfigMap /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:52:36.068: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename configmap STEP: Waiting for a default service account to be provisioned in namespace [It] should be immutable if `immutable` field is set /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/configmap_volume.go:557 [AfterEach] [sig-storage] ConfigMap /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:52:36.111: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "configmap-546" for this suite. • ------------------------------ {"msg":"PASSED [sig-storage] ConfigMap should be immutable if `immutable` field is set","total":-1,"completed":5,"skipped":242,"failed":3,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir-link] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir] One pod requesting one prebound PVC should be able to mount volume and read from pod1"]} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] Projected configMap /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:52:36.276: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename projected STEP: Waiting for a default service account to be provisioned in namespace [It] should be consumable from pods in volume with mappings as non-root with FSGroup [LinuxOnly] [NodeFeature:FSGroup] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/projected_configmap.go:110 STEP: Creating configMap with name projected-configmap-test-volume-map-f07e7404-ce65-4aa4-8d43-e68128cebb71 STEP: Creating a pod to test consume configMaps May 15 01:52:36.309: INFO: Waiting up to 5m0s for pod "pod-projected-configmaps-72a35c55-0e9e-453c-8aca-826d6b9870a1" in namespace "projected-9154" to be "Succeeded or Failed" May 15 01:52:36.312: INFO: Pod "pod-projected-configmaps-72a35c55-0e9e-453c-8aca-826d6b9870a1": Phase="Pending", Reason="", readiness=false. Elapsed: 2.6335ms May 15 01:52:38.314: INFO: Pod "pod-projected-configmaps-72a35c55-0e9e-453c-8aca-826d6b9870a1": Phase="Pending", Reason="", readiness=false. Elapsed: 2.005189243s May 15 01:52:40.317: INFO: Pod "pod-projected-configmaps-72a35c55-0e9e-453c-8aca-826d6b9870a1": Phase="Succeeded", Reason="", readiness=false. Elapsed: 4.008077588s STEP: Saw pod success May 15 01:52:40.317: INFO: Pod "pod-projected-configmaps-72a35c55-0e9e-453c-8aca-826d6b9870a1" satisfied condition "Succeeded or Failed" May 15 01:52:40.319: INFO: Trying to get logs from node node2 pod pod-projected-configmaps-72a35c55-0e9e-453c-8aca-826d6b9870a1 container projected-configmap-volume-test: STEP: delete the pod May 15 01:52:40.333: INFO: Waiting for pod pod-projected-configmaps-72a35c55-0e9e-453c-8aca-826d6b9870a1 to disappear May 15 01:52:40.335: INFO: Pod pod-projected-configmaps-72a35c55-0e9e-453c-8aca-826d6b9870a1 no longer exists [AfterEach] [sig-storage] Projected configMap /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:52:40.335: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "projected-9154" for this suite. • ------------------------------ {"msg":"PASSED [sig-storage] Projected configMap should be consumable from pods in volume with mappings as non-root with FSGroup [LinuxOnly] [NodeFeature:FSGroup]","total":-1,"completed":6,"skipped":325,"failed":3,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir-link] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir] One pod requesting one prebound PVC should be able to mount volume and read from pod1"]} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:52:26.578: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: block] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes STEP: Creating block device on node "node1" using path "/tmp/local-volume-test-0a455b3b-e71e-4027-9117-7d6cfffeeb06" May 15 01:52:30.633: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-0a455b3b-e71e-4027-9117-7d6cfffeeb06 && dd if=/dev/zero of=/tmp/local-volume-test-0a455b3b-e71e-4027-9117-7d6cfffeeb06/file bs=4096 count=5120 && losetup -f /tmp/local-volume-test-0a455b3b-e71e-4027-9117-7d6cfffeeb06/file] Namespace:persistent-local-volumes-test-2167 PodName:hostexec-node1-qjs8r ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:52:30.633: INFO: >>> kubeConfig: /root/.kube/config May 15 01:52:30.941: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c E2E_LOOP_DEV=$(losetup | grep /tmp/local-volume-test-0a455b3b-e71e-4027-9117-7d6cfffeeb06/file | awk '{ print $1 }') 2>&1 > /dev/null && echo ${E2E_LOOP_DEV}] Namespace:persistent-local-volumes-test-2167 PodName:hostexec-node1-qjs8r ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:52:30.941: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:52:31.067: INFO: Creating a PV followed by a PVC May 15 01:52:31.074: INFO: Waiting for PV local-pv9q57k to bind to PVC pvc-4mrtl May 15 01:52:31.075: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-4mrtl] to have phase Bound May 15 01:52:31.077: INFO: PersistentVolumeClaim pvc-4mrtl found but phase is Pending instead of Bound. May 15 01:52:33.079: INFO: PersistentVolumeClaim pvc-4mrtl found but phase is Pending instead of Bound. May 15 01:52:35.082: INFO: PersistentVolumeClaim pvc-4mrtl found but phase is Pending instead of Bound. May 15 01:52:37.086: INFO: PersistentVolumeClaim pvc-4mrtl found but phase is Pending instead of Bound. May 15 01:52:39.089: INFO: PersistentVolumeClaim pvc-4mrtl found but phase is Pending instead of Bound. May 15 01:52:41.094: INFO: PersistentVolumeClaim pvc-4mrtl found but phase is Pending instead of Bound. May 15 01:52:43.096: INFO: PersistentVolumeClaim pvc-4mrtl found and phase=Bound (12.021784892s) May 15 01:52:43.096: INFO: Waiting up to 3m0s for PersistentVolume local-pv9q57k to have phase Bound May 15 01:52:43.098: INFO: PersistentVolume local-pv9q57k found and phase=Bound (1.96868ms) [BeforeEach] Set fsGroup for local volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:257 May 15 01:52:43.102: INFO: We don't set fsGroup on block device, skipped. [AfterEach] [Volume type: block] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:52:43.104: INFO: Deleting PersistentVolumeClaim "pvc-4mrtl" May 15 01:52:43.109: INFO: Deleting PersistentVolume "local-pv9q57k" May 15 01:52:43.113: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c E2E_LOOP_DEV=$(losetup | grep /tmp/local-volume-test-0a455b3b-e71e-4027-9117-7d6cfffeeb06/file | awk '{ print $1 }') 2>&1 > /dev/null && echo ${E2E_LOOP_DEV}] Namespace:persistent-local-volumes-test-2167 PodName:hostexec-node1-qjs8r ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:52:43.113: INFO: >>> kubeConfig: /root/.kube/config STEP: Tear down block device "/dev/loop2" on node "node1" at path /tmp/local-volume-test-0a455b3b-e71e-4027-9117-7d6cfffeeb06/file May 15 01:52:43.404: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c losetup -d /dev/loop2] Namespace:persistent-local-volumes-test-2167 PodName:hostexec-node1-qjs8r ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:52:43.404: INFO: >>> kubeConfig: /root/.kube/config STEP: Removing the test directory /tmp/local-volume-test-0a455b3b-e71e-4027-9117-7d6cfffeeb06 May 15 01:52:43.549: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-0a455b3b-e71e-4027-9117-7d6cfffeeb06] Namespace:persistent-local-volumes-test-2167 PodName:hostexec-node1-qjs8r ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:52:43.550: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:52:44.042: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-2167" for this suite. S [SKIPPING] in Spec Setup (BeforeEach) [17.472 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: block] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 Set fsGroup for local volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:256 should set different fsGroup for second pod if first pod is deleted [BeforeEach] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:282 We don't set fsGroup on block device, skipped. /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:259 ------------------------------ SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes GCEPD /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:52:44.186: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename pv STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes GCEPD /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-gce.go:77 May 15 01:52:44.207: INFO: Only supported for providers [gce gke] (not skeleton) [AfterEach] [sig-storage] PersistentVolumes GCEPD /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:52:44.208: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "pv-6099" for this suite. [AfterEach] [sig-storage] PersistentVolumes GCEPD /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-gce.go:110 May 15 01:52:44.217: INFO: AfterEach: Cleaning up test resources May 15 01:52:44.217: INFO: pvc is nil May 15 01:52:44.217: INFO: pv is nil S [SKIPPING] in Spec Setup (BeforeEach) [0.030 seconds] [sig-storage] PersistentVolumes GCEPD /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 should test that deleting the PV before the pod does not cause pod deletion to fail on PD detach [BeforeEach] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-gce.go:141 Only supported for providers [gce gke] (not skeleton) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-gce.go:85 ------------------------------ SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:50:37.273: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: dir-link] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes May 15 01:50:41.326: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir /tmp/local-volume-test-8cecbb88-1624-42d8-8091-ca767fb1b190-backend && ln -s /tmp/local-volume-test-8cecbb88-1624-42d8-8091-ca767fb1b190-backend /tmp/local-volume-test-8cecbb88-1624-42d8-8091-ca767fb1b190] Namespace:persistent-local-volumes-test-583 PodName:hostexec-node1-ktptr ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:50:41.326: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:50:41.732: INFO: Creating a PV followed by a PVC May 15 01:50:41.740: INFO: Waiting for PV local-pvvhh99 to bind to PVC pvc-kwlwp May 15 01:50:41.740: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-kwlwp] to have phase Bound May 15 01:50:41.742: INFO: PersistentVolumeClaim pvc-kwlwp found but phase is Pending instead of Bound. May 15 01:50:43.745: INFO: PersistentVolumeClaim pvc-kwlwp found and phase=Bound (2.005046006s) May 15 01:50:43.745: INFO: Waiting up to 3m0s for PersistentVolume local-pvvhh99 to have phase Bound May 15 01:50:43.747: INFO: PersistentVolume local-pvvhh99 found and phase=Bound (1.850148ms) [BeforeEach] One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:211 STEP: Creating pod1 STEP: Creating a pod May 15 01:52:43.770: FAIL: Unexpected error: <*errors.errorString | 0xc0030bd090>: { s: "pod \"pod-9566f4e9-ff2d-489c-94bf-709e28a35a7d\" is not Running: timed out waiting for the condition", } pod "pod-9566f4e9-ff2d-489c-94bf-709e28a35a7d" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage.glob..func20.2.3.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 +0x119 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000683980) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc000683980) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc000683980, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:223 STEP: Deleting pod1 STEP: Deleting pod pod-9566f4e9-ff2d-489c-94bf-709e28a35a7d in namespace persistent-local-volumes-test-583 [AfterEach] [Volume type: dir-link] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:52:43.776: INFO: Deleting PersistentVolumeClaim "pvc-kwlwp" May 15 01:52:43.780: INFO: Deleting PersistentVolume "local-pvvhh99" STEP: Removing the test directory May 15 01:52:43.783: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-8cecbb88-1624-42d8-8091-ca767fb1b190 && rm -r /tmp/local-volume-test-8cecbb88-1624-42d8-8091-ca767fb1b190-backend] Namespace:persistent-local-volumes-test-583 PodName:hostexec-node1-ktptr ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:52:43.783: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "persistent-local-volumes-test-583". STEP: Found 15 events. May 15 01:52:44.055: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for hostexec-node1-ktptr: { } Scheduled: Successfully assigned persistent-local-volumes-test-583/hostexec-node1-ktptr to node1 May 15 01:52:44.055: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-9566f4e9-ff2d-489c-94bf-709e28a35a7d: { } Scheduled: Successfully assigned persistent-local-volumes-test-583/pod-9566f4e9-ff2d-489c-94bf-709e28a35a7d to node1 May 15 01:52:44.055: INFO: At 2021-05-15 01:50:38 +0000 UTC - event for hostexec-node1-ktptr: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 15 01:52:44.055: INFO: At 2021-05-15 01:50:39 +0000 UTC - event for hostexec-node1-ktptr: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 1.539326666s May 15 01:52:44.055: INFO: At 2021-05-15 01:50:39 +0000 UTC - event for hostexec-node1-ktptr: {kubelet node1} Created: Created container agnhost-container May 15 01:52:44.055: INFO: At 2021-05-15 01:50:39 +0000 UTC - event for hostexec-node1-ktptr: {kubelet node1} Started: Started container agnhost-container May 15 01:52:44.055: INFO: At 2021-05-15 01:50:41 +0000 UTC - event for pvc-kwlwp: {persistentvolume-controller } ProvisioningFailed: no volume plugin matched name: kubernetes.io/no-provisioner May 15 01:52:44.055: INFO: At 2021-05-15 01:50:45 +0000 UTC - event for pod-9566f4e9-ff2d-489c-94bf-709e28a35a7d: {multus } AddedInterface: Add eth0 [10.244.3.180/24] May 15 01:52:44.055: INFO: At 2021-05-15 01:50:45 +0000 UTC - event for pod-9566f4e9-ff2d-489c-94bf-709e28a35a7d: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:52:44.055: INFO: At 2021-05-15 01:50:46 +0000 UTC - event for pod-9566f4e9-ff2d-489c-94bf-709e28a35a7d: {kubelet node1} Failed: Error: ErrImagePull May 15 01:52:44.055: INFO: At 2021-05-15 01:50:46 +0000 UTC - event for pod-9566f4e9-ff2d-489c-94bf-709e28a35a7d: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:52:44.055: INFO: At 2021-05-15 01:50:47 +0000 UTC - event for pod-9566f4e9-ff2d-489c-94bf-709e28a35a7d: {kubelet node1} SandboxChanged: Pod sandbox changed, it will be killed and re-created. May 15 01:52:44.055: INFO: At 2021-05-15 01:50:49 +0000 UTC - event for pod-9566f4e9-ff2d-489c-94bf-709e28a35a7d: {multus } AddedInterface: Add eth0 [10.244.3.181/24] May 15 01:52:44.055: INFO: At 2021-05-15 01:50:49 +0000 UTC - event for pod-9566f4e9-ff2d-489c-94bf-709e28a35a7d: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 15 01:52:44.055: INFO: At 2021-05-15 01:50:49 +0000 UTC - event for pod-9566f4e9-ff2d-489c-94bf-709e28a35a7d: {kubelet node1} Failed: Error: ImagePullBackOff May 15 01:52:44.057: INFO: POD NODE PHASE GRACE CONDITIONS May 15 01:52:44.057: INFO: hostexec-node1-ktptr node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:50:37 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:50:39 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:50:39 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:50:37 +0000 UTC }] May 15 01:52:44.057: INFO: pod-9566f4e9-ff2d-489c-94bf-709e28a35a7d node1 Pending 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:50:43 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:50:43 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:50:43 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:50:43 +0000 UTC }] May 15 01:52:44.057: INFO: May 15 01:52:44.061: INFO: Logging node info for node master1 May 15 01:52:44.064: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 160612 0 2021-05-14 19:56:35 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:39 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:39 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:39 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:52:39 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[@ :],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:52:44.064: INFO: Logging kubelet events for node master1 May 15 01:52:44.069: INFO: Logging pods the kubelet thinks is on node master1 May 15 01:52:44.078: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.078: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:52:44.078: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:52:44.078: INFO: Init container install-cni ready: true, restart count 0 May 15 01:52:44.078: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:52:44.078: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:52:44.078: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:52:44.078: INFO: Container node-exporter ready: true, restart count 0 May 15 01:52:44.078: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.078: INFO: Container kube-scheduler ready: true, restart count 0 May 15 01:52:44.078: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.078: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:52:44.078: INFO: kubernetes-metrics-scraper-678c97765c-fswrn started at 2021-05-15 00:18:49 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.078: INFO: Container kubernetes-metrics-scraper ready: true, restart count 0 May 15 01:52:44.078: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.078: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:52:44.078: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.078: INFO: Container kube-multus ready: true, restart count 1 May 15 01:52:44.078: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 15 01:52:44.078: INFO: Container docker-registry ready: true, restart count 0 May 15 01:52:44.078: INFO: Container nginx ready: true, restart count 0 May 15 01:52:44.078: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.078: INFO: Container nfd-controller ready: true, restart count 0 W0515 01:52:44.091696 35 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:52:44.117: INFO: Latency metrics for node master1 May 15 01:52:44.117: INFO: Logging node info for node master2 May 15 01:52:44.119: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 160552 0 2021-05-14 19:57:04 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:37 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:37 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:37 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:52:37 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:52:44.120: INFO: Logging kubelet events for node master2 May 15 01:52:44.124: INFO: Logging pods the kubelet thinks is on node master2 May 15 01:52:44.133: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.133: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:52:44.133: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.133: INFO: Container kube-multus ready: true, restart count 1 May 15 01:52:44.133: INFO: coredns-7677f9bb54-96w24 started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.133: INFO: Container coredns ready: true, restart count 0 May 15 01:52:44.133: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:52:44.133: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:52:44.133: INFO: Container node-exporter ready: true, restart count 0 May 15 01:52:44.133: INFO: prometheus-operator-5bb8cb9d8f-fqb87 started at 2021-05-15 00:18:49 +0000 UTC (0+2 container statuses recorded) May 15 01:52:44.133: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:52:44.133: INFO: Container prometheus-operator ready: true, restart count 0 May 15 01:52:44.133: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.133: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:52:44.133: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.134: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:52:44.134: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.134: INFO: Container kube-scheduler ready: true, restart count 2 May 15 01:52:44.134: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:52:44.134: INFO: Init container install-cni ready: true, restart count 0 May 15 01:52:44.134: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:52:44.134: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.134: INFO: Container autoscaler ready: true, restart count 2 W0515 01:52:44.146360 35 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:52:44.169: INFO: Latency metrics for node master2 May 15 01:52:44.169: INFO: Logging node info for node master3 May 15 01:52:44.171: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 160436 0 2021-05-14 19:57:14 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234767872 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324603904 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:35 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:35 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:35 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:52:35 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:52:44.172: INFO: Logging kubelet events for node master3 May 15 01:52:44.176: INFO: Logging pods the kubelet thinks is on node master3 May 15 01:52:44.183: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.183: INFO: Container kube-scheduler ready: true, restart count 3 May 15 01:52:44.183: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.183: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:52:44.183: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.183: INFO: Container kube-proxy ready: true, restart count 1 May 15 01:52:44.183: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:52:44.183: INFO: Init container install-cni ready: true, restart count 0 May 15 01:52:44.183: INFO: Container kube-flannel ready: true, restart count 2 May 15 01:52:44.183: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.183: INFO: Container kube-multus ready: true, restart count 1 May 15 01:52:44.183: INFO: coredns-7677f9bb54-rpj8c started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.183: INFO: Container coredns ready: true, restart count 0 May 15 01:52:44.183: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:52:44.183: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:52:44.183: INFO: Container node-exporter ready: true, restart count 0 May 15 01:52:44.183: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.183: INFO: Container kube-controller-manager ready: true, restart count 3 W0515 01:52:44.194990 35 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:52:44.224: INFO: Latency metrics for node master3 May 15 01:52:44.224: INFO: Logging node info for node node1 May 15 01:52:44.227: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 160833 0 2021-05-14 19:58:21 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1602":"csi-mock-csi-mock-volumes-1602","csi-mock-csi-mock-volumes-253":"csi-mock-csi-mock-volumes-253","csi-mock-csi-mock-volumes-2993":"csi-mock-csi-mock-volumes-2993","csi-mock-csi-mock-volumes-6288":"csi-mock-csi-mock-volumes-6288","csi-mock-csi-mock-volumes-7734":"csi-mock-csi-mock-volumes-7734","csi-mock-csi-mock-volumes-895":"csi-mock-csi-mock-volumes-895","csi-mock-csi-mock-volumes-9728":"csi-mock-csi-mock-volumes-9728","csi-mock-csi-mock-volumes-9809":"csi-mock-csi-mock-volumes-9809"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:00 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:50:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:51:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:36 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:36 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:36 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:52:36 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[@ :],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:52:44.228: INFO: Logging kubelet events for node node1 May 15 01:52:44.232: INFO: Logging pods the kubelet thinks is on node node1 May 15 01:52:44.254: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.254: INFO: Container kube-multus ready: true, restart count 1 May 15 01:52:44.254: INFO: prometheus-k8s-0 started at 2021-05-15 00:19:01 +0000 UTC (0+5 container statuses recorded) May 15 01:52:44.254: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 15 01:52:44.254: INFO: Container grafana ready: true, restart count 0 May 15 01:52:44.254: INFO: Container prometheus ready: true, restart count 26 May 15 01:52:44.254: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 15 01:52:44.254: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 15 01:52:44.254: INFO: csi-mockplugin-resizer-0 started at 2021-05-15 01:52:40 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.254: INFO: Container csi-resizer ready: false, restart count 0 May 15 01:52:44.254: INFO: node-exporter-flvqz started at 2021-05-15 00:18:55 +0000 UTC (0+2 container statuses recorded) May 15 01:52:44.254: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:52:44.254: INFO: Container node-exporter ready: true, restart count 0 May 15 01:52:44.254: INFO: csi-mockplugin-attacher-0 started at 2021-05-15 01:52:36 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.254: INFO: Container csi-attacher ready: true, restart count 0 May 15 01:52:44.254: INFO: pod-bfba8786-cc42-4bcf-babe-8aa8c1a6f88f started at 2021-05-15 01:51:46 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.254: INFO: Container write-pod ready: false, restart count 0 May 15 01:52:44.254: INFO: csi-mockplugin-attacher-0 started at 2021-05-15 01:52:40 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.254: INFO: Container csi-attacher ready: false, restart count 0 May 15 01:52:44.254: INFO: hostexec-node1-ktptr started at 2021-05-15 01:50:37 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.254: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:52:44.254: INFO: csi-mockplugin-0 started at 2021-05-15 01:52:36 +0000 UTC (0+3 container statuses recorded) May 15 01:52:44.254: INFO: Container csi-provisioner ready: false, restart count 0 May 15 01:52:44.254: INFO: Container driver-registrar ready: false, restart count 0 May 15 01:52:44.254: INFO: Container mock ready: false, restart count 0 May 15 01:52:44.254: INFO: cmk-4s6dm started at 2021-05-15 00:18:54 +0000 UTC (0+2 container statuses recorded) May 15 01:52:44.254: INFO: Container nodereport ready: true, restart count 0 May 15 01:52:44.254: INFO: Container reconcile ready: true, restart count 0 May 15 01:52:44.254: INFO: pod-9566f4e9-ff2d-489c-94bf-709e28a35a7d started at 2021-05-15 01:50:43 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.254: INFO: Container write-pod ready: false, restart count 0 May 15 01:52:44.254: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.254: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:52:44.254: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4vxcc started at 2021-05-15 00:19:00 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.254: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:52:44.254: INFO: hostexec-node1-vlzvd started at 2021-05-15 01:51:39 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.254: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:52:44.254: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.254: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:52:44.254: INFO: hostexec-node1-c8b5r started at 2021-05-15 01:52:05 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.254: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:52:44.254: INFO: pod-83dd1c20-b22c-4938-b3e4-dcd2105d871b started at 2021-05-15 01:52:13 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.254: INFO: Container write-pod ready: false, restart count 0 May 15 01:52:44.254: INFO: pvc-volume-tester-x8hbh started at 2021-05-15 01:52:36 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.254: INFO: Container volume-tester ready: true, restart count 0 May 15 01:52:44.254: INFO: hostexec-node1-zbxmq started at 2021-05-15 01:50:51 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.254: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:52:44.254: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:52:44.254: INFO: Init container install-cni ready: true, restart count 0 May 15 01:52:44.255: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:52:44.255: INFO: node-feature-discovery-worker-bw8zg started at 2021-05-15 00:18:56 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.255: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:52:44.255: INFO: collectd-mrzps started at 2021-05-15 00:19:22 +0000 UTC (0+3 container statuses recorded) May 15 01:52:44.255: INFO: Container collectd ready: true, restart count 0 May 15 01:52:44.255: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:52:44.255: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:52:44.255: INFO: pod-11760848-bbb5-4ae1-af82-a7a2a4af6a9c started at 2021-05-15 01:50:59 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.255: INFO: Container write-pod ready: false, restart count 0 May 15 01:52:44.255: INFO: csi-mockplugin-0 started at 2021-05-15 01:52:40 +0000 UTC (0+3 container statuses recorded) May 15 01:52:44.255: INFO: Container csi-provisioner ready: false, restart count 0 May 15 01:52:44.255: INFO: Container driver-registrar ready: false, restart count 0 May 15 01:52:44.255: INFO: Container mock ready: false, restart count 0 May 15 01:52:44.255: INFO: hostexec-node1-qjs8r started at 2021-05-15 01:52:26 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.255: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:52:44.255: INFO: csi-mockplugin-0 started at 2021-05-15 01:52:29 +0000 UTC (0+3 container statuses recorded) May 15 01:52:44.255: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:52:44.255: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:52:44.255: INFO: Container mock ready: true, restart count 0 W0515 01:52:44.266127 35 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:52:44.337: INFO: Latency metrics for node node1 May 15 01:52:44.337: INFO: Logging node info for node node2 May 15 01:52:44.339: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 160757 0 2021-05-14 19:58:22 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1663":"csi-mock-csi-mock-volumes-1663","csi-mock-csi-mock-volumes-3052":"csi-mock-csi-mock-volumes-3052","csi-mock-csi-mock-volumes-5200":"csi-mock-csi-mock-volumes-5200","csi-mock-csi-mock-volumes-5678":"csi-mock-csi-mock-volumes-5678","csi-mock-csi-mock-volumes-8760":"csi-mock-csi-mock-volumes-8760","csi-mock-csi-mock-volumes-9624":"csi-mock-csi-mock-volumes-9624"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:46 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:52:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:52:21 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:io.kubernetes.storage.mock/node":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:41 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:41 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:41 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:52:41 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:52:44.340: INFO: Logging kubelet events for node node2 May 15 01:52:44.344: INFO: Logging pods the kubelet thinks is on node node2 May 15 01:52:44.357: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.357: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:52:44.357: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 15 01:52:44.357: INFO: Container nodereport ready: true, restart count 0 May 15 01:52:44.357: INFO: Container reconcile ready: true, restart count 0 May 15 01:52:44.358: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.358: INFO: Container cmk-webhook ready: true, restart count 0 May 15 01:52:44.358: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 15 01:52:44.358: INFO: Container discover ready: false, restart count 0 May 15 01:52:44.358: INFO: Container init ready: false, restart count 0 May 15 01:52:44.358: INFO: Container install ready: false, restart count 0 May 15 01:52:44.358: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.358: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:52:44.358: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.358: INFO: Container kube-multus ready: true, restart count 1 May 15 01:52:44.358: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.358: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 15 01:52:44.358: INFO: metadata-volume-17b048af-a63a-4616-85a7-d1026d94276b started at (0+0 container statuses recorded) May 15 01:52:44.358: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.358: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:52:44.358: INFO: csi-mockplugin-0 started at 2021-05-15 01:52:00 +0000 UTC (0+3 container statuses recorded) May 15 01:52:44.358: INFO: Container csi-provisioner ready: false, restart count 0 May 15 01:52:44.358: INFO: Container driver-registrar ready: false, restart count 0 May 15 01:52:44.358: INFO: Container mock ready: false, restart count 0 May 15 01:52:44.358: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:52:44.358: INFO: Init container install-cni ready: true, restart count 1 May 15 01:52:44.358: INFO: Container kube-flannel ready: true, restart count 4 May 15 01:52:44.358: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:52:44.358: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:52:44.358: INFO: Container node-exporter ready: true, restart count 0 May 15 01:52:44.358: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 15 01:52:44.358: INFO: Container tas-controller ready: true, restart count 0 May 15 01:52:44.358: INFO: Container tas-extender ready: true, restart count 0 May 15 01:52:44.358: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 15 01:52:44.358: INFO: Container collectd ready: true, restart count 0 May 15 01:52:44.358: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:52:44.358: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:52:44.358: INFO: csi-mockplugin-attacher-0 started at 2021-05-15 01:52:00 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.358: INFO: Container csi-attacher ready: false, restart count 0 May 15 01:52:44.358: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 15 01:52:44.358: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:52:44.358: INFO: pod-subpath-test-configmap-lp9m started at 2021-05-15 01:50:05 +0000 UTC (1+2 container statuses recorded) May 15 01:52:44.358: INFO: Init container init-volume-configmap-lp9m ready: false, restart count 0 May 15 01:52:44.358: INFO: Container test-container-subpath-configmap-lp9m ready: false, restart count 0 May 15 01:52:44.358: INFO: Container test-container-volume-configmap-lp9m ready: false, restart count 0 W0515 01:52:44.370910 35 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:52:44.414: INFO: Latency metrics for node node2 May 15 01:52:44.414: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-583" for this suite. • Failure in Spec Setup (BeforeEach) [127.149 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: dir-link] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:205 should be able to mount volume and read from pod1 [BeforeEach] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:228 May 15 01:52:43.770: Unexpected error: <*errors.errorString | 0xc0030bd090>: { s: "pod \"pod-9566f4e9-ff2d-489c-94bf-709e28a35a7d\" is not Running: timed out waiting for the condition", } pod "pod-9566f4e9-ff2d-489c-94bf-709e28a35a7d" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 ------------------------------ {"msg":"FAILED [sig-storage] PersistentVolumes-local [Volume type: dir-link] One pod requesting one prebound PVC should be able to mount volume and read from pod1","total":-1,"completed":2,"skipped":502,"failed":3,"failures":["[sig-storage] PersistentVolumes-local [Volume type: tmpfs] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir-link] One pod requesting one prebound PVC should be able to mount volume and read from pod1"]} SS ------------------------------ [BeforeEach] [sig-storage] Downward API volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:52:44.304: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename downward-api STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] Downward API volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/downwardapi_volume.go:42 [It] should provide podname as non-root with fsgroup and defaultMode [LinuxOnly] [NodeFeature:FSGroup] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/downwardapi_volume.go:107 STEP: Creating a pod to test downward API volume plugin May 15 01:52:44.336: INFO: Waiting up to 5m0s for pod "metadata-volume-17b048af-a63a-4616-85a7-d1026d94276b" in namespace "downward-api-4110" to be "Succeeded or Failed" May 15 01:52:44.338: INFO: Pod "metadata-volume-17b048af-a63a-4616-85a7-d1026d94276b": Phase="Pending", Reason="", readiness=false. Elapsed: 2.015132ms May 15 01:52:46.340: INFO: Pod "metadata-volume-17b048af-a63a-4616-85a7-d1026d94276b": Phase="Pending", Reason="", readiness=false. Elapsed: 2.004741804s May 15 01:52:48.345: INFO: Pod "metadata-volume-17b048af-a63a-4616-85a7-d1026d94276b": Phase="Succeeded", Reason="", readiness=false. Elapsed: 4.009178662s STEP: Saw pod success May 15 01:52:48.345: INFO: Pod "metadata-volume-17b048af-a63a-4616-85a7-d1026d94276b" satisfied condition "Succeeded or Failed" May 15 01:52:48.348: INFO: Trying to get logs from node node2 pod metadata-volume-17b048af-a63a-4616-85a7-d1026d94276b container client-container: STEP: delete the pod May 15 01:52:48.362: INFO: Waiting for pod metadata-volume-17b048af-a63a-4616-85a7-d1026d94276b to disappear May 15 01:52:48.364: INFO: Pod metadata-volume-17b048af-a63a-4616-85a7-d1026d94276b no longer exists [AfterEach] [sig-storage] Downward API volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:52:48.364: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "downward-api-4110" for this suite. • ------------------------------ {"msg":"PASSED [sig-storage] Downward API volume should provide podname as non-root with fsgroup and defaultMode [LinuxOnly] [NodeFeature:FSGroup]","total":-1,"completed":6,"skipped":891,"failed":2,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] One pod requesting one prebound PVC should be able to mount volume and write from pod1","[sig-storage] PersistentVolumes-local [Volume type: blockfswithformat] One pod requesting one prebound PVC should be able to mount volume and write from pod1"]} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] HostPath /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:52:44.435: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename hostpath STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] HostPath /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/host_path.go:37 [It] should support subPath [NodeConformance] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/host_path.go:93 STEP: Creating a pod to test hostPath subPath May 15 01:52:44.466: INFO: Waiting up to 5m0s for pod "pod-host-path-test" in namespace "hostpath-4373" to be "Succeeded or Failed" May 15 01:52:44.469: INFO: Pod "pod-host-path-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2.280588ms May 15 01:52:46.471: INFO: Pod "pod-host-path-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2.004843956s May 15 01:52:48.474: INFO: Pod "pod-host-path-test": Phase="Pending", Reason="", readiness=false. Elapsed: 4.007988152s May 15 01:52:50.477: INFO: Pod "pod-host-path-test": Phase="Succeeded", Reason="", readiness=false. Elapsed: 6.010417831s STEP: Saw pod success May 15 01:52:50.477: INFO: Pod "pod-host-path-test" satisfied condition "Succeeded or Failed" May 15 01:52:50.479: INFO: Trying to get logs from node node2 pod pod-host-path-test container test-container-2: STEP: delete the pod May 15 01:52:50.492: INFO: Waiting for pod pod-host-path-test to disappear May 15 01:52:50.494: INFO: Pod pod-host-path-test no longer exists [AfterEach] [sig-storage] HostPath /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:52:50.494: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "hostpath-4373" for this suite. • [SLOW TEST:6.066 seconds] [sig-storage] HostPath /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/host_path.go:34 should support subPath [NodeConformance] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/host_path.go:93 ------------------------------ {"msg":"PASSED [sig-storage] HostPath should support subPath [NodeConformance]","total":-1,"completed":3,"skipped":504,"failed":3,"failures":["[sig-storage] PersistentVolumes-local [Volume type: tmpfs] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir-link] One pod requesting one prebound PVC should be able to mount volume and read from pod1"]} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-scheduling] Multi-AZ Cluster Volumes [sig-storage] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:52:50.575: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename multi-az STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-scheduling] Multi-AZ Cluster Volumes [sig-storage] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduling/ubernetes_lite_volumes.go:44 May 15 01:52:50.594: INFO: Only supported for providers [gce gke] (not skeleton) [AfterEach] [sig-scheduling] Multi-AZ Cluster Volumes [sig-storage] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:52:50.596: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "multi-az-6081" for this suite. S [SKIPPING] in Spec Setup (BeforeEach) [0.028 seconds] [sig-scheduling] Multi-AZ Cluster Volumes [sig-storage] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduling/framework.go:40 should schedule pods in the same zones as statically provisioned PVs [BeforeEach] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduling/ubernetes_lite_volumes.go:55 Only supported for providers [gce gke] (not skeleton) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduling/ubernetes_lite_volumes.go:45 ------------------------------ SSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:50:51.460: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: dir-link-bindmounted] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes May 15 01:50:57.514: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir /tmp/local-volume-test-da814d75-0285-4f43-a6fe-8c75b3aa6ae9-backend && mount --bind /tmp/local-volume-test-da814d75-0285-4f43-a6fe-8c75b3aa6ae9-backend /tmp/local-volume-test-da814d75-0285-4f43-a6fe-8c75b3aa6ae9-backend && ln -s /tmp/local-volume-test-da814d75-0285-4f43-a6fe-8c75b3aa6ae9-backend /tmp/local-volume-test-da814d75-0285-4f43-a6fe-8c75b3aa6ae9] Namespace:persistent-local-volumes-test-3351 PodName:hostexec-node1-zbxmq ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:50:57.514: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:50:57.753: INFO: Creating a PV followed by a PVC May 15 01:50:57.761: INFO: Waiting for PV local-pvwtzsp to bind to PVC pvc-tmbp6 May 15 01:50:57.761: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-tmbp6] to have phase Bound May 15 01:50:57.763: INFO: PersistentVolumeClaim pvc-tmbp6 found but phase is Pending instead of Bound. May 15 01:50:59.767: INFO: PersistentVolumeClaim pvc-tmbp6 found and phase=Bound (2.006559639s) May 15 01:50:59.767: INFO: Waiting up to 3m0s for PersistentVolume local-pvwtzsp to have phase Bound May 15 01:50:59.770: INFO: PersistentVolume local-pvwtzsp found and phase=Bound (2.758033ms) [BeforeEach] One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:211 STEP: Creating pod1 STEP: Creating a pod May 15 01:52:59.795: FAIL: Unexpected error: <*errors.errorString | 0xc003ab4450>: { s: "pod \"pod-11760848-bbb5-4ae1-af82-a7a2a4af6a9c\" is not Running: timed out waiting for the condition", } pod "pod-11760848-bbb5-4ae1-af82-a7a2a4af6a9c" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage.glob..func20.2.3.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 +0x119 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc0026cca80) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc0026cca80) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc0026cca80, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:223 STEP: Deleting pod1 STEP: Deleting pod pod-11760848-bbb5-4ae1-af82-a7a2a4af6a9c in namespace persistent-local-volumes-test-3351 [AfterEach] [Volume type: dir-link-bindmounted] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:52:59.804: INFO: Deleting PersistentVolumeClaim "pvc-tmbp6" May 15 01:52:59.808: INFO: Deleting PersistentVolume "local-pvwtzsp" STEP: Removing the test directory May 15 01:52:59.812: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm /tmp/local-volume-test-da814d75-0285-4f43-a6fe-8c75b3aa6ae9 && umount /tmp/local-volume-test-da814d75-0285-4f43-a6fe-8c75b3aa6ae9-backend && rm -r /tmp/local-volume-test-da814d75-0285-4f43-a6fe-8c75b3aa6ae9-backend] Namespace:persistent-local-volumes-test-3351 PodName:hostexec-node1-zbxmq ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:52:59.812: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "persistent-local-volumes-test-3351". STEP: Found 12 events. May 15 01:53:00.067: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for hostexec-node1-zbxmq: { } Scheduled: Successfully assigned persistent-local-volumes-test-3351/hostexec-node1-zbxmq to node1 May 15 01:53:00.067: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-11760848-bbb5-4ae1-af82-a7a2a4af6a9c: { } Scheduled: Successfully assigned persistent-local-volumes-test-3351/pod-11760848-bbb5-4ae1-af82-a7a2a4af6a9c to node1 May 15 01:53:00.067: INFO: At 2021-05-15 01:50:54 +0000 UTC - event for hostexec-node1-zbxmq: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 15 01:53:00.067: INFO: At 2021-05-15 01:50:56 +0000 UTC - event for hostexec-node1-zbxmq: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 1.31788552s May 15 01:53:00.067: INFO: At 2021-05-15 01:50:56 +0000 UTC - event for hostexec-node1-zbxmq: {kubelet node1} Created: Created container agnhost-container May 15 01:53:00.067: INFO: At 2021-05-15 01:50:56 +0000 UTC - event for hostexec-node1-zbxmq: {kubelet node1} Started: Started container agnhost-container May 15 01:53:00.067: INFO: At 2021-05-15 01:51:01 +0000 UTC - event for pod-11760848-bbb5-4ae1-af82-a7a2a4af6a9c: {multus } AddedInterface: Add eth0 [10.244.3.183/24] May 15 01:53:00.067: INFO: At 2021-05-15 01:51:01 +0000 UTC - event for pod-11760848-bbb5-4ae1-af82-a7a2a4af6a9c: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:53:00.067: INFO: At 2021-05-15 01:51:02 +0000 UTC - event for pod-11760848-bbb5-4ae1-af82-a7a2a4af6a9c: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:53:00.067: INFO: At 2021-05-15 01:51:02 +0000 UTC - event for pod-11760848-bbb5-4ae1-af82-a7a2a4af6a9c: {kubelet node1} Failed: Error: ErrImagePull May 15 01:53:00.067: INFO: At 2021-05-15 01:51:02 +0000 UTC - event for pod-11760848-bbb5-4ae1-af82-a7a2a4af6a9c: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 15 01:53:00.067: INFO: At 2021-05-15 01:51:02 +0000 UTC - event for pod-11760848-bbb5-4ae1-af82-a7a2a4af6a9c: {kubelet node1} Failed: Error: ImagePullBackOff May 15 01:53:00.070: INFO: POD NODE PHASE GRACE CONDITIONS May 15 01:53:00.070: INFO: hostexec-node1-zbxmq node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:50:51 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:50:57 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:50:57 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:50:51 +0000 UTC }] May 15 01:53:00.070: INFO: pod-11760848-bbb5-4ae1-af82-a7a2a4af6a9c node1 Pending 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:50:59 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:50:59 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:50:59 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:50:59 +0000 UTC }] May 15 01:53:00.070: INFO: May 15 01:53:00.074: INFO: Logging node info for node master1 May 15 01:53:00.076: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 161270 0 2021-05-14 19:56:35 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:59 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:59 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:59 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:52:59 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[@ :],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:53:00.077: INFO: Logging kubelet events for node master1 May 15 01:53:00.082: INFO: Logging pods the kubelet thinks is on node master1 May 15 01:53:00.091: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.091: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:53:00.091: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:53:00.091: INFO: Init container install-cni ready: true, restart count 0 May 15 01:53:00.091: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:53:00.091: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:53:00.091: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:53:00.091: INFO: Container node-exporter ready: true, restart count 0 May 15 01:53:00.091: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.091: INFO: Container kube-scheduler ready: true, restart count 0 May 15 01:53:00.091: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.091: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:53:00.091: INFO: kubernetes-metrics-scraper-678c97765c-fswrn started at 2021-05-15 00:18:49 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.091: INFO: Container kubernetes-metrics-scraper ready: true, restart count 0 May 15 01:53:00.091: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.091: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:53:00.091: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.091: INFO: Container kube-multus ready: true, restart count 1 May 15 01:53:00.091: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 15 01:53:00.091: INFO: Container docker-registry ready: true, restart count 0 May 15 01:53:00.091: INFO: Container nginx ready: true, restart count 0 May 15 01:53:00.091: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.091: INFO: Container nfd-controller ready: true, restart count 0 W0515 01:53:00.104856 30 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:53:00.137: INFO: Latency metrics for node master1 May 15 01:53:00.137: INFO: Logging node info for node master2 May 15 01:53:00.140: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 161208 0 2021-05-14 19:57:04 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:57 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:57 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:57 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:52:57 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:53:00.140: INFO: Logging kubelet events for node master2 May 15 01:53:00.145: INFO: Logging pods the kubelet thinks is on node master2 May 15 01:53:00.156: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.156: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:53:00.156: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.156: INFO: Container kube-scheduler ready: true, restart count 2 May 15 01:53:00.156: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:53:00.156: INFO: Init container install-cni ready: true, restart count 0 May 15 01:53:00.156: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:53:00.156: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.156: INFO: Container autoscaler ready: true, restart count 2 May 15 01:53:00.156: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:53:00.156: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:53:00.156: INFO: Container node-exporter ready: true, restart count 0 May 15 01:53:00.156: INFO: prometheus-operator-5bb8cb9d8f-fqb87 started at 2021-05-15 00:18:49 +0000 UTC (0+2 container statuses recorded) May 15 01:53:00.156: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:53:00.156: INFO: Container prometheus-operator ready: true, restart count 0 May 15 01:53:00.156: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.156: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:53:00.156: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.156: INFO: Container kube-multus ready: true, restart count 1 May 15 01:53:00.156: INFO: coredns-7677f9bb54-96w24 started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.156: INFO: Container coredns ready: true, restart count 0 May 15 01:53:00.156: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.156: INFO: Container kube-proxy ready: true, restart count 2 W0515 01:53:00.169266 30 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:53:00.195: INFO: Latency metrics for node master2 May 15 01:53:00.195: INFO: Logging node info for node master3 May 15 01:53:00.198: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 161166 0 2021-05-14 19:57:14 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234767872 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324603904 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:55 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:55 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:55 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:52:55 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:53:00.199: INFO: Logging kubelet events for node master3 May 15 01:53:00.203: INFO: Logging pods the kubelet thinks is on node master3 May 15 01:53:00.212: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.212: INFO: Container kube-multus ready: true, restart count 1 May 15 01:53:00.212: INFO: coredns-7677f9bb54-rpj8c started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.212: INFO: Container coredns ready: true, restart count 0 May 15 01:53:00.212: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:53:00.212: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:53:00.212: INFO: Container node-exporter ready: true, restart count 0 May 15 01:53:00.212: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.212: INFO: Container kube-controller-manager ready: true, restart count 3 May 15 01:53:00.212: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.212: INFO: Container kube-scheduler ready: true, restart count 3 May 15 01:53:00.212: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.212: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:53:00.212: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.212: INFO: Container kube-proxy ready: true, restart count 1 May 15 01:53:00.212: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:53:00.212: INFO: Init container install-cni ready: true, restart count 0 May 15 01:53:00.212: INFO: Container kube-flannel ready: true, restart count 2 W0515 01:53:00.224413 30 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:53:00.252: INFO: Latency metrics for node master3 May 15 01:53:00.252: INFO: Logging node info for node node1 May 15 01:53:00.256: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 161204 0 2021-05-14 19:58:21 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1602":"csi-mock-csi-mock-volumes-1602","csi-mock-csi-mock-volumes-253":"csi-mock-csi-mock-volumes-253","csi-mock-csi-mock-volumes-2591":"csi-mock-csi-mock-volumes-2591","csi-mock-csi-mock-volumes-2993":"csi-mock-csi-mock-volumes-2993","csi-mock-csi-mock-volumes-6288":"csi-mock-csi-mock-volumes-6288","csi-mock-csi-mock-volumes-7734":"csi-mock-csi-mock-volumes-7734","csi-mock-csi-mock-volumes-895":"csi-mock-csi-mock-volumes-895","csi-mock-csi-mock-volumes-9474":"csi-mock-csi-mock-volumes-9474","csi-mock-csi-mock-volumes-9728":"csi-mock-csi-mock-volumes-9728","csi-mock-csi-mock-volumes-9809":"csi-mock-csi-mock-volumes-9809"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:00 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:52:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}},"f:status":{"f:volumesAttached":{}}}} {kubelet Update v1 2021-05-15 01:52:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}},"f:volumesInUse":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:56 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:56 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:56 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:52:56 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[@ :],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-6288^4 kubernetes.io/csi/csi-mock-csi-mock-volumes-9474^4],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-6288^4,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-9474^4,DevicePath:,},},Config:nil,},} May 15 01:53:00.256: INFO: Logging kubelet events for node node1 May 15 01:53:00.262: INFO: Logging pods the kubelet thinks is on node node1 May 15 01:53:00.284: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.284: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:53:00.284: INFO: hostexec-node1-c8b5r started at 2021-05-15 01:52:05 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.284: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:53:00.284: INFO: pod-83dd1c20-b22c-4938-b3e4-dcd2105d871b started at 2021-05-15 01:52:13 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.284: INFO: Container write-pod ready: false, restart count 0 May 15 01:53:00.284: INFO: csi-mockplugin-attacher-0 started at 2021-05-15 01:52:50 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.284: INFO: Container csi-attacher ready: true, restart count 0 May 15 01:53:00.284: INFO: hostexec-node1-zbxmq started at 2021-05-15 01:50:51 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.284: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:53:00.284: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:53:00.284: INFO: Init container install-cni ready: true, restart count 0 May 15 01:53:00.284: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:53:00.284: INFO: node-feature-discovery-worker-bw8zg started at 2021-05-15 00:18:56 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.284: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:53:00.284: INFO: collectd-mrzps started at 2021-05-15 00:19:22 +0000 UTC (0+3 container statuses recorded) May 15 01:53:00.284: INFO: Container collectd ready: true, restart count 0 May 15 01:53:00.284: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:53:00.284: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:53:00.284: INFO: pod-11760848-bbb5-4ae1-af82-a7a2a4af6a9c started at 2021-05-15 01:50:59 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.284: INFO: Container write-pod ready: false, restart count 0 May 15 01:53:00.284: INFO: csi-mockplugin-0 started at 2021-05-15 01:52:40 +0000 UTC (0+3 container statuses recorded) May 15 01:53:00.284: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:53:00.284: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:53:00.284: INFO: Container mock ready: true, restart count 0 May 15 01:53:00.284: INFO: csi-mockplugin-0 started at 2021-05-15 01:52:29 +0000 UTC (0+3 container statuses recorded) May 15 01:53:00.284: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:53:00.284: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:53:00.284: INFO: Container mock ready: true, restart count 0 May 15 01:53:00.284: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.284: INFO: Container kube-multus ready: true, restart count 1 May 15 01:53:00.284: INFO: prometheus-k8s-0 started at 2021-05-15 00:19:01 +0000 UTC (0+5 container statuses recorded) May 15 01:53:00.284: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 15 01:53:00.285: INFO: Container grafana ready: true, restart count 0 May 15 01:53:00.285: INFO: Container prometheus ready: true, restart count 26 May 15 01:53:00.285: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 15 01:53:00.285: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 15 01:53:00.285: INFO: csi-mockplugin-resizer-0 started at 2021-05-15 01:52:40 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.285: INFO: Container csi-resizer ready: true, restart count 0 May 15 01:53:00.285: INFO: pvc-volume-tester-vxhg6 started at 2021-05-15 01:52:47 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.285: INFO: Container volume-tester ready: false, restart count 0 May 15 01:53:00.285: INFO: pod-aebbf8b7-c03c-4180-9a91-0c9b975a3efc started at 2021-05-15 01:52:59 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.285: INFO: Container write-pod ready: false, restart count 0 May 15 01:53:00.285: INFO: node-exporter-flvqz started at 2021-05-15 00:18:55 +0000 UTC (0+2 container statuses recorded) May 15 01:53:00.285: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:53:00.285: INFO: Container node-exporter ready: true, restart count 0 May 15 01:53:00.285: INFO: hostexec-node1-bdvv2 started at 2021-05-15 01:52:48 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.285: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:53:00.285: INFO: csi-mockplugin-attacher-0 started at 2021-05-15 01:52:36 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.285: INFO: Container csi-attacher ready: true, restart count 0 May 15 01:53:00.285: INFO: pod-bfba8786-cc42-4bcf-babe-8aa8c1a6f88f started at 2021-05-15 01:51:46 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.285: INFO: Container write-pod ready: false, restart count 0 May 15 01:53:00.285: INFO: csi-mockplugin-attacher-0 started at 2021-05-15 01:52:40 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.285: INFO: Container csi-attacher ready: true, restart count 0 May 15 01:53:00.285: INFO: csi-mockplugin-0 started at 2021-05-15 01:52:36 +0000 UTC (0+3 container statuses recorded) May 15 01:53:00.285: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:53:00.285: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:53:00.285: INFO: Container mock ready: true, restart count 0 May 15 01:53:00.285: INFO: cmk-4s6dm started at 2021-05-15 00:18:54 +0000 UTC (0+2 container statuses recorded) May 15 01:53:00.285: INFO: Container nodereport ready: true, restart count 0 May 15 01:53:00.285: INFO: Container reconcile ready: true, restart count 0 May 15 01:53:00.285: INFO: csi-mockplugin-0 started at 2021-05-15 01:52:50 +0000 UTC (0+3 container statuses recorded) May 15 01:53:00.285: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:53:00.285: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:53:00.285: INFO: Container mock ready: true, restart count 0 May 15 01:53:00.285: INFO: pvc-volume-tester-v2hgd started at 2021-05-15 01:52:52 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.285: INFO: Container volume-tester ready: false, restart count 0 May 15 01:53:00.285: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.285: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:53:00.285: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4vxcc started at 2021-05-15 00:19:00 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.285: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:53:00.285: INFO: hostexec-node1-vlzvd started at 2021-05-15 01:51:39 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.285: INFO: Container agnhost-container ready: true, restart count 0 W0515 01:53:00.296793 30 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:53:00.410: INFO: Latency metrics for node node1 May 15 01:53:00.410: INFO: Logging node info for node node2 May 15 01:53:00.413: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 161107 0 2021-05-14 19:58:22 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1663":"csi-mock-csi-mock-volumes-1663","csi-mock-csi-mock-volumes-3052":"csi-mock-csi-mock-volumes-3052","csi-mock-csi-mock-volumes-5200":"csi-mock-csi-mock-volumes-5200","csi-mock-csi-mock-volumes-5678":"csi-mock-csi-mock-volumes-5678","csi-mock-csi-mock-volumes-8760":"csi-mock-csi-mock-volumes-8760","csi-mock-csi-mock-volumes-9624":"csi-mock-csi-mock-volumes-9624"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:46 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:52:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:52:21 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:io.kubernetes.storage.mock/node":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:51 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:51 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:52:51 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:52:51 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:53:00.414: INFO: Logging kubelet events for node node2 May 15 01:53:00.418: INFO: Logging pods the kubelet thinks is on node node2 May 15 01:53:00.429: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:53:00.429: INFO: Init container install-cni ready: true, restart count 1 May 15 01:53:00.429: INFO: Container kube-flannel ready: true, restart count 4 May 15 01:53:00.429: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:53:00.429: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:53:00.429: INFO: Container node-exporter ready: true, restart count 0 May 15 01:53:00.429: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 15 01:53:00.429: INFO: Container tas-controller ready: true, restart count 0 May 15 01:53:00.429: INFO: Container tas-extender ready: true, restart count 0 May 15 01:53:00.429: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 15 01:53:00.429: INFO: Container collectd ready: true, restart count 0 May 15 01:53:00.429: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:53:00.429: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:53:00.429: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.429: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:53:00.429: INFO: pod-subpath-test-configmap-lp9m started at 2021-05-15 01:50:05 +0000 UTC (1+2 container statuses recorded) May 15 01:53:00.429: INFO: Init container init-volume-configmap-lp9m ready: false, restart count 0 May 15 01:53:00.429: INFO: Container test-container-subpath-configmap-lp9m ready: false, restart count 0 May 15 01:53:00.429: INFO: Container test-container-volume-configmap-lp9m ready: false, restart count 0 May 15 01:53:00.429: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.429: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:53:00.429: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 15 01:53:00.429: INFO: Container nodereport ready: true, restart count 0 May 15 01:53:00.429: INFO: Container reconcile ready: true, restart count 0 May 15 01:53:00.429: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.429: INFO: Container cmk-webhook ready: true, restart count 0 May 15 01:53:00.429: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 15 01:53:00.429: INFO: Container discover ready: false, restart count 0 May 15 01:53:00.429: INFO: Container init ready: false, restart count 0 May 15 01:53:00.429: INFO: Container install ready: false, restart count 0 May 15 01:53:00.429: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.429: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:53:00.429: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.429: INFO: Container kube-multus ready: true, restart count 1 May 15 01:53:00.429: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.429: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 15 01:53:00.429: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:53:00.429: INFO: Container nginx-proxy ready: true, restart count 2 W0515 01:53:00.442567 30 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:53:00.477: INFO: Latency metrics for node node2 May 15 01:53:00.477: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-3351" for this suite. • Failure in Spec Setup (BeforeEach) [129.025 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: dir-link-bindmounted] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:205 should be able to mount volume and write from pod1 [BeforeEach] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:234 May 15 01:52:59.795: Unexpected error: <*errors.errorString | 0xc003ab4450>: { s: "pod \"pod-11760848-bbb5-4ae1-af82-a7a2a4af6a9c\" is not Running: timed out waiting for the condition", } pod "pod-11760848-bbb5-4ae1-af82-a7a2a4af6a9c" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 ------------------------------ [BeforeEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:52:00.397: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename csi-mock-volumes STEP: Waiting for a default service account to be provisioned in namespace [It] should preserve attachment policy when no CSIDriver present /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:274 STEP: Building a driver namespace object, basename csi-mock-volumes-5200 STEP: Waiting for a default service account to be provisioned in namespace STEP: deploying csi mock driver May 15 01:52:00.483: INFO: creating *v1.ServiceAccount: csi-mock-volumes-5200-2087/csi-attacher May 15 01:52:00.485: INFO: creating *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-5200 May 15 01:52:00.485: INFO: Define cluster role external-attacher-runner-csi-mock-volumes-5200 May 15 01:52:00.489: INFO: creating *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-5200 May 15 01:52:00.492: INFO: creating *v1.Role: csi-mock-volumes-5200-2087/external-attacher-cfg-csi-mock-volumes-5200 May 15 01:52:00.495: INFO: creating *v1.RoleBinding: csi-mock-volumes-5200-2087/csi-attacher-role-cfg May 15 01:52:00.498: INFO: creating *v1.ServiceAccount: csi-mock-volumes-5200-2087/csi-provisioner May 15 01:52:00.501: INFO: creating *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-5200 May 15 01:52:00.501: INFO: Define cluster role external-provisioner-runner-csi-mock-volumes-5200 May 15 01:52:00.503: INFO: creating *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-5200 May 15 01:52:00.506: INFO: creating *v1.Role: csi-mock-volumes-5200-2087/external-provisioner-cfg-csi-mock-volumes-5200 May 15 01:52:00.510: INFO: creating *v1.RoleBinding: csi-mock-volumes-5200-2087/csi-provisioner-role-cfg May 15 01:52:00.513: INFO: creating *v1.ServiceAccount: csi-mock-volumes-5200-2087/csi-resizer May 15 01:52:00.516: INFO: creating *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-5200 May 15 01:52:00.516: INFO: Define cluster role external-resizer-runner-csi-mock-volumes-5200 May 15 01:52:00.519: INFO: creating *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-5200 May 15 01:52:00.522: INFO: creating *v1.Role: csi-mock-volumes-5200-2087/external-resizer-cfg-csi-mock-volumes-5200 May 15 01:52:00.524: INFO: creating *v1.RoleBinding: csi-mock-volumes-5200-2087/csi-resizer-role-cfg May 15 01:52:00.526: INFO: creating *v1.ServiceAccount: csi-mock-volumes-5200-2087/csi-mock May 15 01:52:00.529: INFO: creating *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-5200 May 15 01:52:00.531: INFO: creating *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-5200 May 15 01:52:00.533: INFO: creating *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-5200 May 15 01:52:00.536: INFO: creating *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-5200 May 15 01:52:00.538: INFO: creating *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-5200 May 15 01:52:00.540: INFO: creating *v1.StorageClass: csi-mock-sc-csi-mock-volumes-5200 May 15 01:52:00.543: INFO: creating *v1.StatefulSet: csi-mock-volumes-5200-2087/csi-mockplugin May 15 01:52:00.547: INFO: creating *v1.StatefulSet: csi-mock-volumes-5200-2087/csi-mockplugin-attacher May 15 01:52:00.551: INFO: waiting for CSIDriver csi-mock-csi-mock-volumes-5200 to register on node node2 STEP: Creating pod May 15 01:52:05.563: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil May 15 01:52:05.569: INFO: Waiting up to 5m0s for PersistentVolumeClaims [pvc-csqgh] to have phase Bound May 15 01:52:05.571: INFO: PersistentVolumeClaim pvc-csqgh found but phase is Pending instead of Bound. May 15 01:52:07.575: INFO: PersistentVolumeClaim pvc-csqgh found and phase=Bound (2.00552284s) STEP: Checking if VolumeAttachment was created for the pod STEP: Deleting pod pvc-volume-tester-rvn7c May 15 01:52:15.602: INFO: Deleting pod "pvc-volume-tester-rvn7c" in namespace "csi-mock-volumes-5200" May 15 01:52:15.607: INFO: Wait up to 5m0s for pod "pvc-volume-tester-rvn7c" to be fully deleted STEP: Deleting claim pvc-csqgh May 15 01:52:29.619: INFO: Waiting up to 2m0s for PersistentVolume pvc-1867165b-bb6e-4dc0-bdbf-c05848f35719 to get deleted May 15 01:52:29.622: INFO: PersistentVolume pvc-1867165b-bb6e-4dc0-bdbf-c05848f35719 found and phase=Bound (2.463034ms) May 15 01:52:31.627: INFO: PersistentVolume pvc-1867165b-bb6e-4dc0-bdbf-c05848f35719 was removed STEP: Deleting storageclass csi-mock-volumes-5200-sc STEP: Cleaning up resources STEP: deleting the test namespace: csi-mock-volumes-5200 STEP: Waiting for namespaces [csi-mock-volumes-5200] to vanish STEP: uninstalling csi mock driver May 15 01:52:37.643: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-5200-2087/csi-attacher May 15 01:52:37.649: INFO: deleting *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-5200 May 15 01:52:37.653: INFO: deleting *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-5200 May 15 01:52:37.657: INFO: deleting *v1.Role: csi-mock-volumes-5200-2087/external-attacher-cfg-csi-mock-volumes-5200 May 15 01:52:37.661: INFO: deleting *v1.RoleBinding: csi-mock-volumes-5200-2087/csi-attacher-role-cfg May 15 01:52:37.666: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-5200-2087/csi-provisioner May 15 01:52:37.670: INFO: deleting *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-5200 May 15 01:52:37.678: INFO: deleting *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-5200 May 15 01:52:37.686: INFO: deleting *v1.Role: csi-mock-volumes-5200-2087/external-provisioner-cfg-csi-mock-volumes-5200 May 15 01:52:37.696: INFO: deleting *v1.RoleBinding: csi-mock-volumes-5200-2087/csi-provisioner-role-cfg May 15 01:52:37.702: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-5200-2087/csi-resizer May 15 01:52:37.706: INFO: deleting *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-5200 May 15 01:52:37.712: INFO: deleting *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-5200 May 15 01:52:37.716: INFO: deleting *v1.Role: csi-mock-volumes-5200-2087/external-resizer-cfg-csi-mock-volumes-5200 May 15 01:52:37.720: INFO: deleting *v1.RoleBinding: csi-mock-volumes-5200-2087/csi-resizer-role-cfg May 15 01:52:37.724: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-5200-2087/csi-mock May 15 01:52:37.730: INFO: deleting *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-5200 May 15 01:52:37.734: INFO: deleting *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-5200 May 15 01:52:37.738: INFO: deleting *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-5200 May 15 01:52:37.742: INFO: deleting *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-5200 May 15 01:52:37.746: INFO: deleting *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-5200 May 15 01:52:37.749: INFO: deleting *v1.StorageClass: csi-mock-sc-csi-mock-volumes-5200 May 15 01:52:37.754: INFO: deleting *v1.StatefulSet: csi-mock-volumes-5200-2087/csi-mockplugin May 15 01:52:37.759: INFO: deleting *v1.StatefulSet: csi-mock-volumes-5200-2087/csi-mockplugin-attacher STEP: deleting the driver namespace: csi-mock-volumes-5200-2087 STEP: Waiting for namespaces [csi-mock-volumes-5200-2087] to vanish [AfterEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:53:05.772: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "csi-mock-volumes-5200" for this suite. May 15 01:53:05.780: INFO: Namespace csi-mock-volumes-5200 was already deleted STEP: Destroying namespace "csi-mock-volumes-5200-2087" for this suite. May 15 01:53:05.783: INFO: Namespace csi-mock-volumes-5200-2087 was already deleted • [SLOW TEST:65.386 seconds] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 CSI attach test using mock driver /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:252 should preserve attachment policy when no CSIDriver present /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:274 ------------------------------ {"msg":"PASSED [sig-storage] CSI mock volume CSI attach test using mock driver should preserve attachment policy when no CSIDriver present","total":-1,"completed":4,"skipped":127,"failed":3,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir-link] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir-link] One pod requesting one prebound PVC should be able to mount volume and write from pod1","[sig-storage] PersistentVolumes-local [Volume type: dir] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2"]} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:52:29.256: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename csi-mock-volumes STEP: Waiting for a default service account to be provisioned in namespace [It] should not require VolumeAttach for drivers without attachment /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:274 STEP: Building a driver namespace object, basename csi-mock-volumes-2993 STEP: Waiting for a default service account to be provisioned in namespace STEP: deploying csi mock driver May 15 01:52:29.335: INFO: creating *v1.ServiceAccount: csi-mock-volumes-2993-6460/csi-attacher May 15 01:52:29.338: INFO: creating *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-2993 May 15 01:52:29.338: INFO: Define cluster role external-attacher-runner-csi-mock-volumes-2993 May 15 01:52:29.341: INFO: creating *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-2993 May 15 01:52:29.344: INFO: creating *v1.Role: csi-mock-volumes-2993-6460/external-attacher-cfg-csi-mock-volumes-2993 May 15 01:52:29.347: INFO: creating *v1.RoleBinding: csi-mock-volumes-2993-6460/csi-attacher-role-cfg May 15 01:52:29.349: INFO: creating *v1.ServiceAccount: csi-mock-volumes-2993-6460/csi-provisioner May 15 01:52:29.352: INFO: creating *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-2993 May 15 01:52:29.352: INFO: Define cluster role external-provisioner-runner-csi-mock-volumes-2993 May 15 01:52:29.354: INFO: creating *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-2993 May 15 01:52:29.357: INFO: creating *v1.Role: csi-mock-volumes-2993-6460/external-provisioner-cfg-csi-mock-volumes-2993 May 15 01:52:29.360: INFO: creating *v1.RoleBinding: csi-mock-volumes-2993-6460/csi-provisioner-role-cfg May 15 01:52:29.363: INFO: creating *v1.ServiceAccount: csi-mock-volumes-2993-6460/csi-resizer May 15 01:52:29.365: INFO: creating *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-2993 May 15 01:52:29.366: INFO: Define cluster role external-resizer-runner-csi-mock-volumes-2993 May 15 01:52:29.368: INFO: creating *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-2993 May 15 01:52:29.371: INFO: creating *v1.Role: csi-mock-volumes-2993-6460/external-resizer-cfg-csi-mock-volumes-2993 May 15 01:52:29.374: INFO: creating *v1.RoleBinding: csi-mock-volumes-2993-6460/csi-resizer-role-cfg May 15 01:52:29.376: INFO: creating *v1.ServiceAccount: csi-mock-volumes-2993-6460/csi-mock May 15 01:52:29.379: INFO: creating *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-2993 May 15 01:52:29.382: INFO: creating *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-2993 May 15 01:52:29.384: INFO: creating *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-2993 May 15 01:52:29.386: INFO: creating *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-2993 May 15 01:52:29.389: INFO: creating *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-2993 May 15 01:52:29.392: INFO: creating *v1.StorageClass: csi-mock-sc-csi-mock-volumes-2993 May 15 01:52:29.394: INFO: creating *v1.StatefulSet: csi-mock-volumes-2993-6460/csi-mockplugin May 15 01:52:29.398: INFO: creating *v1.CSIDriver: csi-mock-csi-mock-volumes-2993 May 15 01:52:29.400: INFO: waiting up to 4m0s for CSIDriver "csi-mock-csi-mock-volumes-2993" May 15 01:52:29.402: INFO: waiting for CSIDriver csi-mock-csi-mock-volumes-2993 to register on node node1 STEP: Creating pod May 15 01:52:34.415: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil May 15 01:52:34.421: INFO: Waiting up to 5m0s for PersistentVolumeClaims [pvc-4vncw] to have phase Bound May 15 01:52:34.423: INFO: PersistentVolumeClaim pvc-4vncw found but phase is Pending instead of Bound. May 15 01:52:36.426: INFO: PersistentVolumeClaim pvc-4vncw found and phase=Bound (2.004866212s) STEP: Checking if VolumeAttachment was created for the pod STEP: Deleting pod pvc-volume-tester-x8hbh May 15 01:52:42.451: INFO: Deleting pod "pvc-volume-tester-x8hbh" in namespace "csi-mock-volumes-2993" May 15 01:52:42.456: INFO: Wait up to 5m0s for pod "pvc-volume-tester-x8hbh" to be fully deleted STEP: Deleting claim pvc-4vncw May 15 01:52:50.470: INFO: Waiting up to 2m0s for PersistentVolume pvc-4a81fd0d-374b-4733-91a9-cbcc764c2697 to get deleted May 15 01:52:50.472: INFO: PersistentVolume pvc-4a81fd0d-374b-4733-91a9-cbcc764c2697 found and phase=Bound (1.922436ms) May 15 01:52:52.476: INFO: PersistentVolume pvc-4a81fd0d-374b-4733-91a9-cbcc764c2697 was removed STEP: Deleting storageclass csi-mock-volumes-2993-sc STEP: Cleaning up resources STEP: deleting the test namespace: csi-mock-volumes-2993 STEP: Waiting for namespaces [csi-mock-volumes-2993] to vanish STEP: uninstalling csi mock driver May 15 01:52:58.494: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-2993-6460/csi-attacher May 15 01:52:58.501: INFO: deleting *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-2993 May 15 01:52:58.506: INFO: deleting *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-2993 May 15 01:52:58.510: INFO: deleting *v1.Role: csi-mock-volumes-2993-6460/external-attacher-cfg-csi-mock-volumes-2993 May 15 01:52:58.515: INFO: deleting *v1.RoleBinding: csi-mock-volumes-2993-6460/csi-attacher-role-cfg May 15 01:52:58.519: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-2993-6460/csi-provisioner May 15 01:52:58.525: INFO: deleting *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-2993 May 15 01:52:58.536: INFO: deleting *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-2993 May 15 01:52:58.547: INFO: deleting *v1.Role: csi-mock-volumes-2993-6460/external-provisioner-cfg-csi-mock-volumes-2993 May 15 01:52:58.551: INFO: deleting *v1.RoleBinding: csi-mock-volumes-2993-6460/csi-provisioner-role-cfg May 15 01:52:58.555: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-2993-6460/csi-resizer May 15 01:52:58.560: INFO: deleting *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-2993 May 15 01:52:58.563: INFO: deleting *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-2993 May 15 01:52:58.567: INFO: deleting *v1.Role: csi-mock-volumes-2993-6460/external-resizer-cfg-csi-mock-volumes-2993 May 15 01:52:58.572: INFO: deleting *v1.RoleBinding: csi-mock-volumes-2993-6460/csi-resizer-role-cfg May 15 01:52:58.576: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-2993-6460/csi-mock May 15 01:52:58.580: INFO: deleting *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-2993 May 15 01:52:58.584: INFO: deleting *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-2993 May 15 01:52:58.588: INFO: deleting *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-2993 May 15 01:52:58.592: INFO: deleting *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-2993 May 15 01:52:58.596: INFO: deleting *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-2993 May 15 01:52:58.600: INFO: deleting *v1.StorageClass: csi-mock-sc-csi-mock-volumes-2993 May 15 01:52:58.604: INFO: deleting *v1.StatefulSet: csi-mock-volumes-2993-6460/csi-mockplugin May 15 01:52:58.611: INFO: deleting *v1.CSIDriver: csi-mock-csi-mock-volumes-2993 STEP: deleting the driver namespace: csi-mock-volumes-2993-6460 STEP: Waiting for namespaces [csi-mock-volumes-2993-6460] to vanish [AfterEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:53:10.627: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "csi-mock-volumes-2993" for this suite. May 15 01:53:10.634: INFO: Namespace csi-mock-volumes-2993 was already deleted STEP: Destroying namespace "csi-mock-volumes-2993-6460" for this suite. May 15 01:53:10.637: INFO: Namespace csi-mock-volumes-2993-6460 was already deleted • [SLOW TEST:41.381 seconds] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 CSI attach test using mock driver /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:252 should not require VolumeAttach for drivers without attachment /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:274 ------------------------------ {"msg":"PASSED [sig-storage] CSI mock volume CSI attach test using mock driver should not require VolumeAttach for drivers without attachment","total":-1,"completed":8,"skipped":1141,"failed":3,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir] One pod requesting one prebound PVC should be able to mount volume and write from pod1","[sig-storage] PersistentVolumes-local [Volume type: tmpfs] One pod requesting one prebound PVC should be able to mount volume and read from pod1","[sig-storage] PersistentVolumes-local [Volume type: dir-link-bindmounted] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2"]} SSSSSSS ------------------------------ [BeforeEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:52:40.424: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename csi-mock-volumes STEP: Waiting for a default service account to be provisioned in namespace [It] should expand volume without restarting pod if nodeExpansion=off /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:455 STEP: Building a driver namespace object, basename csi-mock-volumes-9474 STEP: Waiting for a default service account to be provisioned in namespace STEP: deploying csi mock driver May 15 01:52:40.495: INFO: creating *v1.ServiceAccount: csi-mock-volumes-9474-9531/csi-attacher May 15 01:52:40.498: INFO: creating *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-9474 May 15 01:52:40.498: INFO: Define cluster role external-attacher-runner-csi-mock-volumes-9474 May 15 01:52:40.500: INFO: creating *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-9474 May 15 01:52:40.503: INFO: creating *v1.Role: csi-mock-volumes-9474-9531/external-attacher-cfg-csi-mock-volumes-9474 May 15 01:52:40.506: INFO: creating *v1.RoleBinding: csi-mock-volumes-9474-9531/csi-attacher-role-cfg May 15 01:52:40.510: INFO: creating *v1.ServiceAccount: csi-mock-volumes-9474-9531/csi-provisioner May 15 01:52:40.513: INFO: creating *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-9474 May 15 01:52:40.513: INFO: Define cluster role external-provisioner-runner-csi-mock-volumes-9474 May 15 01:52:40.516: INFO: creating *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-9474 May 15 01:52:40.519: INFO: creating *v1.Role: csi-mock-volumes-9474-9531/external-provisioner-cfg-csi-mock-volumes-9474 May 15 01:52:40.521: INFO: creating *v1.RoleBinding: csi-mock-volumes-9474-9531/csi-provisioner-role-cfg May 15 01:52:40.524: INFO: creating *v1.ServiceAccount: csi-mock-volumes-9474-9531/csi-resizer May 15 01:52:40.526: INFO: creating *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-9474 May 15 01:52:40.526: INFO: Define cluster role external-resizer-runner-csi-mock-volumes-9474 May 15 01:52:40.529: INFO: creating *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-9474 May 15 01:52:40.531: INFO: creating *v1.Role: csi-mock-volumes-9474-9531/external-resizer-cfg-csi-mock-volumes-9474 May 15 01:52:40.534: INFO: creating *v1.RoleBinding: csi-mock-volumes-9474-9531/csi-resizer-role-cfg May 15 01:52:40.536: INFO: creating *v1.ServiceAccount: csi-mock-volumes-9474-9531/csi-mock May 15 01:52:40.539: INFO: creating *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-9474 May 15 01:52:40.542: INFO: creating *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-9474 May 15 01:52:40.545: INFO: creating *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-9474 May 15 01:52:40.547: INFO: creating *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-9474 May 15 01:52:40.549: INFO: creating *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-9474 May 15 01:52:40.552: INFO: creating *v1.StorageClass: csi-mock-sc-csi-mock-volumes-9474 May 15 01:52:40.554: INFO: creating *v1.StatefulSet: csi-mock-volumes-9474-9531/csi-mockplugin May 15 01:52:40.558: INFO: creating *v1.StatefulSet: csi-mock-volumes-9474-9531/csi-mockplugin-attacher May 15 01:52:40.561: INFO: creating *v1.StatefulSet: csi-mock-volumes-9474-9531/csi-mockplugin-resizer May 15 01:52:40.565: INFO: waiting for CSIDriver csi-mock-csi-mock-volumes-9474 to register on node node1 STEP: Creating pod May 15 01:52:50.085: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil May 15 01:52:50.092: INFO: Waiting up to 5m0s for PersistentVolumeClaims [pvc-jfk2r] to have phase Bound May 15 01:52:50.094: INFO: PersistentVolumeClaim pvc-jfk2r found but phase is Pending instead of Bound. May 15 01:52:52.097: INFO: PersistentVolumeClaim pvc-jfk2r found and phase=Bound (2.004615362s) STEP: Expanding current pvc STEP: Waiting for persistent volume resize to finish STEP: Waiting for PVC resize to finish STEP: Deleting pod pvc-volume-tester-v2hgd May 15 01:53:04.135: INFO: Deleting pod "pvc-volume-tester-v2hgd" in namespace "csi-mock-volumes-9474" May 15 01:53:04.141: INFO: Wait up to 5m0s for pod "pvc-volume-tester-v2hgd" to be fully deleted STEP: Deleting claim pvc-jfk2r May 15 01:53:14.154: INFO: Waiting up to 2m0s for PersistentVolume pvc-2d1e48a8-f0d0-4e9e-b7ba-9ba710c1eb76 to get deleted May 15 01:53:14.156: INFO: PersistentVolume pvc-2d1e48a8-f0d0-4e9e-b7ba-9ba710c1eb76 found and phase=Bound (2.453894ms) May 15 01:53:16.164: INFO: PersistentVolume pvc-2d1e48a8-f0d0-4e9e-b7ba-9ba710c1eb76 found and phase=Released (2.010133051s) May 15 01:53:18.167: INFO: PersistentVolume pvc-2d1e48a8-f0d0-4e9e-b7ba-9ba710c1eb76 was removed STEP: Deleting storageclass csi-mock-volumes-9474-sc STEP: Cleaning up resources STEP: deleting the test namespace: csi-mock-volumes-9474 STEP: Waiting for namespaces [csi-mock-volumes-9474] to vanish STEP: uninstalling csi mock driver May 15 01:53:24.186: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-9474-9531/csi-attacher May 15 01:53:24.190: INFO: deleting *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-9474 May 15 01:53:24.195: INFO: deleting *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-9474 May 15 01:53:24.199: INFO: deleting *v1.Role: csi-mock-volumes-9474-9531/external-attacher-cfg-csi-mock-volumes-9474 May 15 01:53:24.204: INFO: deleting *v1.RoleBinding: csi-mock-volumes-9474-9531/csi-attacher-role-cfg May 15 01:53:24.209: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-9474-9531/csi-provisioner May 15 01:53:24.216: INFO: deleting *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-9474 May 15 01:53:24.226: INFO: deleting *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-9474 May 15 01:53:24.234: INFO: deleting *v1.Role: csi-mock-volumes-9474-9531/external-provisioner-cfg-csi-mock-volumes-9474 May 15 01:53:24.239: INFO: deleting *v1.RoleBinding: csi-mock-volumes-9474-9531/csi-provisioner-role-cfg May 15 01:53:24.243: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-9474-9531/csi-resizer May 15 01:53:24.247: INFO: deleting *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-9474 May 15 01:53:24.251: INFO: deleting *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-9474 May 15 01:53:24.256: INFO: deleting *v1.Role: csi-mock-volumes-9474-9531/external-resizer-cfg-csi-mock-volumes-9474 May 15 01:53:24.260: INFO: deleting *v1.RoleBinding: csi-mock-volumes-9474-9531/csi-resizer-role-cfg May 15 01:53:24.264: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-9474-9531/csi-mock May 15 01:53:24.269: INFO: deleting *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-9474 May 15 01:53:24.274: INFO: deleting *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-9474 May 15 01:53:24.278: INFO: deleting *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-9474 May 15 01:53:24.282: INFO: deleting *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-9474 May 15 01:53:24.287: INFO: deleting *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-9474 May 15 01:53:24.291: INFO: deleting *v1.StorageClass: csi-mock-sc-csi-mock-volumes-9474 May 15 01:53:24.295: INFO: deleting *v1.StatefulSet: csi-mock-volumes-9474-9531/csi-mockplugin May 15 01:53:24.300: INFO: deleting *v1.StatefulSet: csi-mock-volumes-9474-9531/csi-mockplugin-attacher May 15 01:53:24.306: INFO: deleting *v1.StatefulSet: csi-mock-volumes-9474-9531/csi-mockplugin-resizer STEP: deleting the driver namespace: csi-mock-volumes-9474-9531 STEP: Waiting for namespaces [csi-mock-volumes-9474-9531] to vanish [AfterEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:53:36.319: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "csi-mock-volumes-9474" for this suite. May 15 01:53:36.326: INFO: Namespace csi-mock-volumes-9474 was already deleted STEP: Destroying namespace "csi-mock-volumes-9474-9531" for this suite. May 15 01:53:36.329: INFO: Namespace csi-mock-volumes-9474-9531 was already deleted • [SLOW TEST:55.904 seconds] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 CSI Volume expansion /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:426 should expand volume without restarting pod if nodeExpansion=off /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:455 ------------------------------ {"msg":"PASSED [sig-storage] CSI mock volume CSI Volume expansion should expand volume without restarting pod if nodeExpansion=off","total":-1,"completed":7,"skipped":364,"failed":3,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir-link] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir] One pod requesting one prebound PVC should be able to mount volume and read from pod1"]} SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] Volume limits /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:53:36.548: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename volume-limits-on-node STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] Volume limits /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/volume_limits.go:35 May 15 01:53:36.568: INFO: Only supported for providers [aws gce gke] (not skeleton) [AfterEach] [sig-storage] Volume limits /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:53:36.570: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "volume-limits-on-node-7437" for this suite. S [SKIPPING] in Spec Setup (BeforeEach) [0.029 seconds] [sig-storage] Volume limits /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 should verify that all nodes have volume limits [BeforeEach] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/volume_limits.go:41 Only supported for providers [aws gce gke] (not skeleton) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/volume_limits.go:36 ------------------------------ SSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [BeforeEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:52:36.053: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename csi-mock-volumes STEP: Waiting for a default service account to be provisioned in namespace [It] should not be passed when CSIDriver does not exist /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:358 STEP: Building a driver namespace object, basename csi-mock-volumes-6288 STEP: Waiting for a default service account to be provisioned in namespace STEP: deploying csi mock driver May 15 01:52:36.122: INFO: creating *v1.ServiceAccount: csi-mock-volumes-6288-8845/csi-attacher May 15 01:52:36.125: INFO: creating *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-6288 May 15 01:52:36.125: INFO: Define cluster role external-attacher-runner-csi-mock-volumes-6288 May 15 01:52:36.127: INFO: creating *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-6288 May 15 01:52:36.129: INFO: creating *v1.Role: csi-mock-volumes-6288-8845/external-attacher-cfg-csi-mock-volumes-6288 May 15 01:52:36.131: INFO: creating *v1.RoleBinding: csi-mock-volumes-6288-8845/csi-attacher-role-cfg May 15 01:52:36.134: INFO: creating *v1.ServiceAccount: csi-mock-volumes-6288-8845/csi-provisioner May 15 01:52:36.136: INFO: creating *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-6288 May 15 01:52:36.136: INFO: Define cluster role external-provisioner-runner-csi-mock-volumes-6288 May 15 01:52:36.138: INFO: creating *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-6288 May 15 01:52:36.141: INFO: creating *v1.Role: csi-mock-volumes-6288-8845/external-provisioner-cfg-csi-mock-volumes-6288 May 15 01:52:36.143: INFO: creating *v1.RoleBinding: csi-mock-volumes-6288-8845/csi-provisioner-role-cfg May 15 01:52:36.145: INFO: creating *v1.ServiceAccount: csi-mock-volumes-6288-8845/csi-resizer May 15 01:52:36.147: INFO: creating *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-6288 May 15 01:52:36.147: INFO: Define cluster role external-resizer-runner-csi-mock-volumes-6288 May 15 01:52:36.150: INFO: creating *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-6288 May 15 01:52:36.151: INFO: creating *v1.Role: csi-mock-volumes-6288-8845/external-resizer-cfg-csi-mock-volumes-6288 May 15 01:52:36.154: INFO: creating *v1.RoleBinding: csi-mock-volumes-6288-8845/csi-resizer-role-cfg May 15 01:52:36.156: INFO: creating *v1.ServiceAccount: csi-mock-volumes-6288-8845/csi-mock May 15 01:52:36.158: INFO: creating *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-6288 May 15 01:52:36.161: INFO: creating *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-6288 May 15 01:52:36.163: INFO: creating *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-6288 May 15 01:52:36.165: INFO: creating *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-6288 May 15 01:52:36.167: INFO: creating *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-6288 May 15 01:52:36.169: INFO: creating *v1.StorageClass: csi-mock-sc-csi-mock-volumes-6288 May 15 01:52:36.172: INFO: creating *v1.StatefulSet: csi-mock-volumes-6288-8845/csi-mockplugin May 15 01:52:36.176: INFO: creating *v1.StatefulSet: csi-mock-volumes-6288-8845/csi-mockplugin-attacher May 15 01:52:36.179: INFO: waiting for CSIDriver csi-mock-csi-mock-volumes-6288 to register on node node1 STEP: Creating pod May 15 01:52:45.692: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil May 15 01:52:45.699: INFO: Waiting up to 5m0s for PersistentVolumeClaims [pvc-tvqc4] to have phase Bound May 15 01:52:45.701: INFO: PersistentVolumeClaim pvc-tvqc4 found but phase is Pending instead of Bound. May 15 01:52:47.704: INFO: PersistentVolumeClaim pvc-tvqc4 found and phase=Bound (2.005756536s) STEP: Deleting the previously created pod May 15 01:53:13.729: INFO: Deleting pod "pvc-volume-tester-vxhg6" in namespace "csi-mock-volumes-6288" May 15 01:53:13.734: INFO: Wait up to 5m0s for pod "pvc-volume-tester-vxhg6" to be fully deleted STEP: Checking CSI driver logs May 15 01:53:17.752: INFO: Found NodeUnpublishVolume: {json: {"Method":"/csi.v1.Node/NodeUnpublishVolume","Request":{"volume_id":"4","target_path":"/var/lib/kubelet/pods/9f8b3bc4-90b3-4969-bf90-ac1797319916/volumes/kubernetes.io~csi/pvc-27664be3-e05a-40b2-90a4-0a0252243bdf/mount"},"Response":{},"Error":"","FullError":null} Method:NodeUnpublishVolume Request:{VolumeContext:map[]} FullError:{Code:OK Message:} Error:} STEP: Deleting pod pvc-volume-tester-vxhg6 May 15 01:53:17.752: INFO: Deleting pod "pvc-volume-tester-vxhg6" in namespace "csi-mock-volumes-6288" STEP: Deleting claim pvc-tvqc4 May 15 01:53:17.761: INFO: Waiting up to 2m0s for PersistentVolume pvc-27664be3-e05a-40b2-90a4-0a0252243bdf to get deleted May 15 01:53:17.765: INFO: PersistentVolume pvc-27664be3-e05a-40b2-90a4-0a0252243bdf found and phase=Bound (3.167589ms) May 15 01:53:19.768: INFO: PersistentVolume pvc-27664be3-e05a-40b2-90a4-0a0252243bdf was removed STEP: Deleting storageclass csi-mock-volumes-6288-sc STEP: Cleaning up resources STEP: deleting the test namespace: csi-mock-volumes-6288 STEP: Waiting for namespaces [csi-mock-volumes-6288] to vanish STEP: uninstalling csi mock driver May 15 01:53:25.786: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-6288-8845/csi-attacher May 15 01:53:25.790: INFO: deleting *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-6288 May 15 01:53:25.794: INFO: deleting *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-6288 May 15 01:53:25.798: INFO: deleting *v1.Role: csi-mock-volumes-6288-8845/external-attacher-cfg-csi-mock-volumes-6288 May 15 01:53:25.802: INFO: deleting *v1.RoleBinding: csi-mock-volumes-6288-8845/csi-attacher-role-cfg May 15 01:53:25.806: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-6288-8845/csi-provisioner May 15 01:53:25.813: INFO: deleting *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-6288 May 15 01:53:25.822: INFO: deleting *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-6288 May 15 01:53:25.831: INFO: deleting *v1.Role: csi-mock-volumes-6288-8845/external-provisioner-cfg-csi-mock-volumes-6288 May 15 01:53:25.838: INFO: deleting *v1.RoleBinding: csi-mock-volumes-6288-8845/csi-provisioner-role-cfg May 15 01:53:25.842: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-6288-8845/csi-resizer May 15 01:53:25.846: INFO: deleting *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-6288 May 15 01:53:25.858: INFO: deleting *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-6288 May 15 01:53:25.862: INFO: deleting *v1.Role: csi-mock-volumes-6288-8845/external-resizer-cfg-csi-mock-volumes-6288 May 15 01:53:25.866: INFO: deleting *v1.RoleBinding: csi-mock-volumes-6288-8845/csi-resizer-role-cfg May 15 01:53:25.870: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-6288-8845/csi-mock May 15 01:53:25.874: INFO: deleting *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-6288 May 15 01:53:25.879: INFO: deleting *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-6288 May 15 01:53:25.883: INFO: deleting *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-6288 May 15 01:53:25.887: INFO: deleting *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-6288 May 15 01:53:25.891: INFO: deleting *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-6288 May 15 01:53:25.896: INFO: deleting *v1.StorageClass: csi-mock-sc-csi-mock-volumes-6288 May 15 01:53:25.900: INFO: deleting *v1.StatefulSet: csi-mock-volumes-6288-8845/csi-mockplugin May 15 01:53:25.904: INFO: deleting *v1.StatefulSet: csi-mock-volumes-6288-8845/csi-mockplugin-attacher STEP: deleting the driver namespace: csi-mock-volumes-6288-8845 STEP: Waiting for namespaces [csi-mock-volumes-6288-8845] to vanish [AfterEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:53:37.915: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "csi-mock-volumes-6288" for this suite. May 15 01:53:37.922: INFO: Namespace csi-mock-volumes-6288 was already deleted STEP: Destroying namespace "csi-mock-volumes-6288-8845" for this suite. May 15 01:53:37.924: INFO: Namespace csi-mock-volumes-6288-8845 was already deleted • [SLOW TEST:61.871 seconds] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 CSI workload information using mock driver /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:308 should not be passed when CSIDriver does not exist /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:358 ------------------------------ {"msg":"PASSED [sig-storage] CSI mock volume CSI workload information using mock driver should not be passed when CSIDriver does not exist","total":-1,"completed":1,"skipped":101,"failed":4,"failures":["[sig-storage] PersistentVolumes-local [Volume type: blockfswithoutformat] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir-link-bindmounted] One pod requesting one prebound PVC should be able to mount volume and read from pod1","[sig-storage] PersistentVolumes-local [Volume type: blockfswithoutformat] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2"]} SSSSSSS ------------------------------ May 15 01:53:37.942: INFO: Running AfterSuite actions on all nodes [BeforeEach] [sig-storage] EmptyDir volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:53:36.627: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename emptydir STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] when FSGroup is specified [LinuxOnly] [NodeFeature:FSGroup] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/empty_dir.go:47 [It] new files should be created with FSGroup ownership when container is root /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/empty_dir.go:52 STEP: Creating a pod to test emptydir 0644 on tmpfs May 15 01:53:36.664: INFO: Waiting up to 5m0s for pod "pod-f035edeb-674f-4ad9-85a2-4a69411717f3" in namespace "emptydir-2648" to be "Succeeded or Failed" May 15 01:53:36.666: INFO: Pod "pod-f035edeb-674f-4ad9-85a2-4a69411717f3": Phase="Pending", Reason="", readiness=false. Elapsed: 2.745273ms May 15 01:53:38.671: INFO: Pod "pod-f035edeb-674f-4ad9-85a2-4a69411717f3": Phase="Pending", Reason="", readiness=false. Elapsed: 2.006850736s May 15 01:53:40.675: INFO: Pod "pod-f035edeb-674f-4ad9-85a2-4a69411717f3": Phase="Succeeded", Reason="", readiness=false. Elapsed: 4.011664363s STEP: Saw pod success May 15 01:53:40.675: INFO: Pod "pod-f035edeb-674f-4ad9-85a2-4a69411717f3" satisfied condition "Succeeded or Failed" May 15 01:53:40.678: INFO: Trying to get logs from node node2 pod pod-f035edeb-674f-4ad9-85a2-4a69411717f3 container test-container: STEP: delete the pod May 15 01:53:40.691: INFO: Waiting for pod pod-f035edeb-674f-4ad9-85a2-4a69411717f3 to disappear May 15 01:53:40.693: INFO: Pod pod-f035edeb-674f-4ad9-85a2-4a69411717f3 no longer exists [AfterEach] [sig-storage] EmptyDir volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:53:40.693: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "emptydir-2648" for this suite. • ------------------------------ {"msg":"PASSED [sig-storage] EmptyDir volumes when FSGroup is specified [LinuxOnly] [NodeFeature:FSGroup] new files should be created with FSGroup ownership when container is root","total":-1,"completed":8,"skipped":504,"failed":3,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir-link] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir] One pod requesting one prebound PVC should be able to mount volume and read from pod1"]} May 15 01:53:40.703: INFO: Running AfterSuite actions on all nodes [BeforeEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:52:50.657: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename csi-mock-volumes STEP: Waiting for a default service account to be provisioned in namespace [It] should be passed when podInfoOnMount=true /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:358 STEP: Building a driver namespace object, basename csi-mock-volumes-2591 STEP: Waiting for a default service account to be provisioned in namespace STEP: deploying csi mock driver May 15 01:52:50.714: INFO: creating *v1.ServiceAccount: csi-mock-volumes-2591-2321/csi-attacher May 15 01:52:50.717: INFO: creating *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-2591 May 15 01:52:50.718: INFO: Define cluster role external-attacher-runner-csi-mock-volumes-2591 May 15 01:52:50.721: INFO: creating *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-2591 May 15 01:52:50.725: INFO: creating *v1.Role: csi-mock-volumes-2591-2321/external-attacher-cfg-csi-mock-volumes-2591 May 15 01:52:50.727: INFO: creating *v1.RoleBinding: csi-mock-volumes-2591-2321/csi-attacher-role-cfg May 15 01:52:50.730: INFO: creating *v1.ServiceAccount: csi-mock-volumes-2591-2321/csi-provisioner May 15 01:52:50.733: INFO: creating *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-2591 May 15 01:52:50.733: INFO: Define cluster role external-provisioner-runner-csi-mock-volumes-2591 May 15 01:52:50.736: INFO: creating *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-2591 May 15 01:52:50.739: INFO: creating *v1.Role: csi-mock-volumes-2591-2321/external-provisioner-cfg-csi-mock-volumes-2591 May 15 01:52:50.741: INFO: creating *v1.RoleBinding: csi-mock-volumes-2591-2321/csi-provisioner-role-cfg May 15 01:52:50.743: INFO: creating *v1.ServiceAccount: csi-mock-volumes-2591-2321/csi-resizer May 15 01:52:50.746: INFO: creating *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-2591 May 15 01:52:50.746: INFO: Define cluster role external-resizer-runner-csi-mock-volumes-2591 May 15 01:52:50.749: INFO: creating *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-2591 May 15 01:52:50.751: INFO: creating *v1.Role: csi-mock-volumes-2591-2321/external-resizer-cfg-csi-mock-volumes-2591 May 15 01:52:50.754: INFO: creating *v1.RoleBinding: csi-mock-volumes-2591-2321/csi-resizer-role-cfg May 15 01:52:50.757: INFO: creating *v1.ServiceAccount: csi-mock-volumes-2591-2321/csi-mock May 15 01:52:50.759: INFO: creating *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-2591 May 15 01:52:50.762: INFO: creating *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-2591 May 15 01:52:50.765: INFO: creating *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-2591 May 15 01:52:50.767: INFO: creating *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-2591 May 15 01:52:50.770: INFO: creating *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-2591 May 15 01:52:50.772: INFO: creating *v1.StorageClass: csi-mock-sc-csi-mock-volumes-2591 May 15 01:52:50.778: INFO: creating *v1.StatefulSet: csi-mock-volumes-2591-2321/csi-mockplugin May 15 01:52:50.785: INFO: creating *v1.CSIDriver: csi-mock-csi-mock-volumes-2591 May 15 01:52:50.789: INFO: creating *v1.StatefulSet: csi-mock-volumes-2591-2321/csi-mockplugin-attacher May 15 01:52:50.794: INFO: waiting up to 4m0s for CSIDriver "csi-mock-csi-mock-volumes-2591" May 15 01:52:50.799: INFO: waiting for CSIDriver csi-mock-csi-mock-volumes-2591 to register on node node1 STEP: Creating pod May 15 01:53:00.315: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil May 15 01:53:00.320: INFO: Waiting up to 5m0s for PersistentVolumeClaims [pvc-cc8zh] to have phase Bound May 15 01:53:00.323: INFO: PersistentVolumeClaim pvc-cc8zh found but phase is Pending instead of Bound. May 15 01:53:02.325: INFO: PersistentVolumeClaim pvc-cc8zh found and phase=Bound (2.00479771s) STEP: checking for CSIInlineVolumes feature May 15 01:53:16.360: INFO: Pod inline-volume-tqppx has the following logs: May 15 01:53:16.360: INFO: Deleting pod "inline-volume-tqppx" in namespace "csi-mock-volumes-2591" May 15 01:53:16.365: INFO: Wait up to 5m0s for pod "inline-volume-tqppx" to be fully deleted STEP: Deleting the previously created pod May 15 01:53:18.369: INFO: Deleting pod "pvc-volume-tester-2lwns" in namespace "csi-mock-volumes-2591" May 15 01:53:18.374: INFO: Wait up to 5m0s for pod "pvc-volume-tester-2lwns" to be fully deleted STEP: Checking CSI driver logs May 15 01:53:32.387: INFO: Found volume attribute csi.storage.k8s.io/ephemeral: false May 15 01:53:32.387: INFO: Found volume attribute csi.storage.k8s.io/serviceAccount.name: default May 15 01:53:32.387: INFO: Found volume attribute csi.storage.k8s.io/pod.name: pvc-volume-tester-2lwns May 15 01:53:32.387: INFO: Found volume attribute csi.storage.k8s.io/pod.namespace: csi-mock-volumes-2591 May 15 01:53:32.387: INFO: Found volume attribute csi.storage.k8s.io/pod.uid: 3470a51b-5ef9-471c-976f-3c74838590bc May 15 01:53:32.387: INFO: Found NodeUnpublishVolume: {json: {"Method":"/csi.v1.Node/NodeUnpublishVolume","Request":{"volume_id":"4","target_path":"/var/lib/kubelet/pods/3470a51b-5ef9-471c-976f-3c74838590bc/volumes/kubernetes.io~csi/pvc-f6d54aba-87f8-4275-9052-d986763dc7e5/mount"},"Response":{},"Error":"","FullError":null} Method:NodeUnpublishVolume Request:{VolumeContext:map[]} FullError:{Code:OK Message:} Error:} STEP: Deleting pod pvc-volume-tester-2lwns May 15 01:53:32.387: INFO: Deleting pod "pvc-volume-tester-2lwns" in namespace "csi-mock-volumes-2591" STEP: Deleting claim pvc-cc8zh May 15 01:53:32.399: INFO: Waiting up to 2m0s for PersistentVolume pvc-f6d54aba-87f8-4275-9052-d986763dc7e5 to get deleted May 15 01:53:32.401: INFO: PersistentVolume pvc-f6d54aba-87f8-4275-9052-d986763dc7e5 found and phase=Bound (2.176408ms) May 15 01:53:34.404: INFO: PersistentVolume pvc-f6d54aba-87f8-4275-9052-d986763dc7e5 was removed STEP: Deleting storageclass csi-mock-volumes-2591-sc STEP: Cleaning up resources STEP: deleting the test namespace: csi-mock-volumes-2591 STEP: Waiting for namespaces [csi-mock-volumes-2591] to vanish STEP: uninstalling csi mock driver May 15 01:53:40.426: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-2591-2321/csi-attacher May 15 01:53:40.430: INFO: deleting *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-2591 May 15 01:53:40.435: INFO: deleting *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-2591 May 15 01:53:40.439: INFO: deleting *v1.Role: csi-mock-volumes-2591-2321/external-attacher-cfg-csi-mock-volumes-2591 May 15 01:53:40.443: INFO: deleting *v1.RoleBinding: csi-mock-volumes-2591-2321/csi-attacher-role-cfg May 15 01:53:40.449: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-2591-2321/csi-provisioner May 15 01:53:40.460: INFO: deleting *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-2591 May 15 01:53:40.472: INFO: deleting *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-2591 May 15 01:53:40.478: INFO: deleting *v1.Role: csi-mock-volumes-2591-2321/external-provisioner-cfg-csi-mock-volumes-2591 May 15 01:53:40.482: INFO: deleting *v1.RoleBinding: csi-mock-volumes-2591-2321/csi-provisioner-role-cfg May 15 01:53:40.486: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-2591-2321/csi-resizer May 15 01:53:40.491: INFO: deleting *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-2591 May 15 01:53:40.495: INFO: deleting *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-2591 May 15 01:53:40.499: INFO: deleting *v1.Role: csi-mock-volumes-2591-2321/external-resizer-cfg-csi-mock-volumes-2591 May 15 01:53:40.504: INFO: deleting *v1.RoleBinding: csi-mock-volumes-2591-2321/csi-resizer-role-cfg May 15 01:53:40.508: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-2591-2321/csi-mock May 15 01:53:40.512: INFO: deleting *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-2591 May 15 01:53:40.516: INFO: deleting *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-2591 May 15 01:53:40.520: INFO: deleting *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-2591 May 15 01:53:40.524: INFO: deleting *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-2591 May 15 01:53:40.528: INFO: deleting *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-2591 May 15 01:53:40.532: INFO: deleting *v1.StorageClass: csi-mock-sc-csi-mock-volumes-2591 May 15 01:53:40.536: INFO: deleting *v1.StatefulSet: csi-mock-volumes-2591-2321/csi-mockplugin May 15 01:53:40.542: INFO: deleting *v1.CSIDriver: csi-mock-csi-mock-volumes-2591 May 15 01:53:40.549: INFO: deleting *v1.StatefulSet: csi-mock-volumes-2591-2321/csi-mockplugin-attacher STEP: deleting the driver namespace: csi-mock-volumes-2591-2321 STEP: Waiting for namespaces [csi-mock-volumes-2591-2321] to vanish [AfterEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:53:46.563: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "csi-mock-volumes-2591" for this suite. May 15 01:53:46.570: INFO: Namespace csi-mock-volumes-2591 was already deleted STEP: Destroying namespace "csi-mock-volumes-2591-2321" for this suite. May 15 01:53:46.573: INFO: Namespace csi-mock-volumes-2591-2321 was already deleted • [SLOW TEST:55.917 seconds] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 CSI workload information using mock driver /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:308 should be passed when podInfoOnMount=true /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:358 ------------------------------ {"msg":"PASSED [sig-storage] CSI mock volume CSI workload information using mock driver should be passed when podInfoOnMount=true","total":-1,"completed":4,"skipped":563,"failed":3,"failures":["[sig-storage] PersistentVolumes-local [Volume type: tmpfs] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir-link] One pod requesting one prebound PVC should be able to mount volume and read from pod1"]} May 15 01:53:46.575: INFO: Running AfterSuite actions on all nodes [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:51:39.806: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: block] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes STEP: Creating block device on node "node1" using path "/tmp/local-volume-test-85033660-563c-4caa-b173-c4086900f5eb" May 15 01:51:43.859: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-85033660-563c-4caa-b173-c4086900f5eb && dd if=/dev/zero of=/tmp/local-volume-test-85033660-563c-4caa-b173-c4086900f5eb/file bs=4096 count=5120 && losetup -f /tmp/local-volume-test-85033660-563c-4caa-b173-c4086900f5eb/file] Namespace:persistent-local-volumes-test-5959 PodName:hostexec-node1-vlzvd ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:51:43.859: INFO: >>> kubeConfig: /root/.kube/config May 15 01:51:44.038: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c E2E_LOOP_DEV=$(losetup | grep /tmp/local-volume-test-85033660-563c-4caa-b173-c4086900f5eb/file | awk '{ print $1 }') 2>&1 > /dev/null && echo ${E2E_LOOP_DEV}] Namespace:persistent-local-volumes-test-5959 PodName:hostexec-node1-vlzvd ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:51:44.038: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:51:44.161: INFO: Creating a PV followed by a PVC May 15 01:51:44.167: INFO: Waiting for PV local-pvmxcm5 to bind to PVC pvc-gngh6 May 15 01:51:44.167: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-gngh6] to have phase Bound May 15 01:51:44.169: INFO: PersistentVolumeClaim pvc-gngh6 found but phase is Pending instead of Bound. May 15 01:51:46.173: INFO: PersistentVolumeClaim pvc-gngh6 found and phase=Bound (2.005446473s) May 15 01:51:46.173: INFO: Waiting up to 3m0s for PersistentVolume local-pvmxcm5 to have phase Bound May 15 01:51:46.175: INFO: PersistentVolume local-pvmxcm5 found and phase=Bound (1.854808ms) [BeforeEach] One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:211 STEP: Creating pod1 STEP: Creating a pod May 15 01:53:46.198: FAIL: Unexpected error: <*errors.errorString | 0xc003cfcf00>: { s: "pod \"pod-bfba8786-cc42-4bcf-babe-8aa8c1a6f88f\" is not Running: timed out waiting for the condition", } pod "pod-bfba8786-cc42-4bcf-babe-8aa8c1a6f88f" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage.glob..func20.2.3.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 +0x119 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000e53800) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc000e53800) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc000e53800, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:223 STEP: Deleting pod1 STEP: Deleting pod pod-bfba8786-cc42-4bcf-babe-8aa8c1a6f88f in namespace persistent-local-volumes-test-5959 [AfterEach] [Volume type: block] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:53:46.205: INFO: Deleting PersistentVolumeClaim "pvc-gngh6" May 15 01:53:46.209: INFO: Deleting PersistentVolume "local-pvmxcm5" May 15 01:53:46.213: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c E2E_LOOP_DEV=$(losetup | grep /tmp/local-volume-test-85033660-563c-4caa-b173-c4086900f5eb/file | awk '{ print $1 }') 2>&1 > /dev/null && echo ${E2E_LOOP_DEV}] Namespace:persistent-local-volumes-test-5959 PodName:hostexec-node1-vlzvd ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:53:46.213: INFO: >>> kubeConfig: /root/.kube/config STEP: Tear down block device "/dev/loop0" on node "node1" at path /tmp/local-volume-test-85033660-563c-4caa-b173-c4086900f5eb/file May 15 01:53:46.326: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c losetup -d /dev/loop0] Namespace:persistent-local-volumes-test-5959 PodName:hostexec-node1-vlzvd ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:53:46.326: INFO: >>> kubeConfig: /root/.kube/config STEP: Removing the test directory /tmp/local-volume-test-85033660-563c-4caa-b173-c4086900f5eb May 15 01:53:46.453: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-85033660-563c-4caa-b173-c4086900f5eb] Namespace:persistent-local-volumes-test-5959 PodName:hostexec-node1-vlzvd ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:53:46.453: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "persistent-local-volumes-test-5959". STEP: Found 14 events. May 15 01:53:46.585: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for hostexec-node1-vlzvd: { } Scheduled: Successfully assigned persistent-local-volumes-test-5959/hostexec-node1-vlzvd to node1 May 15 01:53:46.585: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-bfba8786-cc42-4bcf-babe-8aa8c1a6f88f: { } Scheduled: Successfully assigned persistent-local-volumes-test-5959/pod-bfba8786-cc42-4bcf-babe-8aa8c1a6f88f to node1 May 15 01:53:46.585: INFO: At 2021-05-15 01:51:40 +0000 UTC - event for hostexec-node1-vlzvd: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 15 01:53:46.585: INFO: At 2021-05-15 01:51:40 +0000 UTC - event for hostexec-node1-vlzvd: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 507.799348ms May 15 01:53:46.585: INFO: At 2021-05-15 01:51:41 +0000 UTC - event for hostexec-node1-vlzvd: {kubelet node1} Created: Created container agnhost-container May 15 01:53:46.585: INFO: At 2021-05-15 01:51:41 +0000 UTC - event for hostexec-node1-vlzvd: {kubelet node1} Started: Started container agnhost-container May 15 01:53:46.585: INFO: At 2021-05-15 01:51:48 +0000 UTC - event for pod-bfba8786-cc42-4bcf-babe-8aa8c1a6f88f: {kubelet node1} SuccessfulMountVolume: MapVolume.MapPodDevice succeeded for volume "local-pvmxcm5" globalMapPath "/var/lib/kubelet/plugins/kubernetes.io~local-volume/volumeDevices/local-pvmxcm5" May 15 01:53:46.585: INFO: At 2021-05-15 01:51:48 +0000 UTC - event for pod-bfba8786-cc42-4bcf-babe-8aa8c1a6f88f: {kubelet node1} SuccessfulMountVolume: MapVolume.MapPodDevice succeeded for volume "local-pvmxcm5" volumeMapPath "/var/lib/kubelet/pods/e38b38b2-79ec-4b87-b060-21493b2b2839/volumeDevices/kubernetes.io~local-volume" May 15 01:53:46.585: INFO: At 2021-05-15 01:51:49 +0000 UTC - event for pod-bfba8786-cc42-4bcf-babe-8aa8c1a6f88f: {multus } AddedInterface: Add eth0 [10.244.3.186/24] May 15 01:53:46.585: INFO: At 2021-05-15 01:51:49 +0000 UTC - event for pod-bfba8786-cc42-4bcf-babe-8aa8c1a6f88f: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:53:46.585: INFO: At 2021-05-15 01:51:50 +0000 UTC - event for pod-bfba8786-cc42-4bcf-babe-8aa8c1a6f88f: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:53:46.585: INFO: At 2021-05-15 01:51:50 +0000 UTC - event for pod-bfba8786-cc42-4bcf-babe-8aa8c1a6f88f: {kubelet node1} Failed: Error: ErrImagePull May 15 01:53:46.585: INFO: At 2021-05-15 01:51:51 +0000 UTC - event for pod-bfba8786-cc42-4bcf-babe-8aa8c1a6f88f: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 15 01:53:46.585: INFO: At 2021-05-15 01:51:51 +0000 UTC - event for pod-bfba8786-cc42-4bcf-babe-8aa8c1a6f88f: {kubelet node1} Failed: Error: ImagePullBackOff May 15 01:53:46.587: INFO: POD NODE PHASE GRACE CONDITIONS May 15 01:53:46.587: INFO: hostexec-node1-vlzvd node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:51:39 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:51:41 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:51:41 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:51:39 +0000 UTC }] May 15 01:53:46.588: INFO: pod-bfba8786-cc42-4bcf-babe-8aa8c1a6f88f node1 Pending 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:51:46 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:51:46 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:51:46 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:51:46 +0000 UTC }] May 15 01:53:46.588: INFO: May 15 01:53:46.592: INFO: Logging node info for node master1 May 15 01:53:46.595: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 162001 0 2021-05-14 19:56:35 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:53:39 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:53:39 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:53:39 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:53:39 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[@ :],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:53:46.595: INFO: Logging kubelet events for node master1 May 15 01:53:46.601: INFO: Logging pods the kubelet thinks is on node master1 May 15 01:53:46.610: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.610: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:53:46.610: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:53:46.610: INFO: Init container install-cni ready: true, restart count 0 May 15 01:53:46.610: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:53:46.610: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:53:46.610: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:53:46.610: INFO: Container node-exporter ready: true, restart count 0 May 15 01:53:46.610: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.610: INFO: Container kube-scheduler ready: true, restart count 0 May 15 01:53:46.610: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 15 01:53:46.611: INFO: Container docker-registry ready: true, restart count 0 May 15 01:53:46.611: INFO: Container nginx ready: true, restart count 0 May 15 01:53:46.611: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.611: INFO: Container nfd-controller ready: true, restart count 0 May 15 01:53:46.611: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.611: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:53:46.611: INFO: kubernetes-metrics-scraper-678c97765c-fswrn started at 2021-05-15 00:18:49 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.611: INFO: Container kubernetes-metrics-scraper ready: true, restart count 0 May 15 01:53:46.611: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.611: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:53:46.611: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.611: INFO: Container kube-multus ready: true, restart count 1 W0515 01:53:46.624014 28 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:53:46.649: INFO: Latency metrics for node master1 May 15 01:53:46.649: INFO: Logging node info for node master2 May 15 01:53:46.652: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 161979 0 2021-05-14 19:57:04 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:53:37 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:53:37 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:53:37 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:53:37 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:53:46.652: INFO: Logging kubelet events for node master2 May 15 01:53:46.657: INFO: Logging pods the kubelet thinks is on node master2 May 15 01:53:46.666: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.666: INFO: Container kube-scheduler ready: true, restart count 2 May 15 01:53:46.666: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:53:46.666: INFO: Init container install-cni ready: true, restart count 0 May 15 01:53:46.666: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:53:46.666: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.666: INFO: Container autoscaler ready: true, restart count 2 May 15 01:53:46.666: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:53:46.666: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:53:46.666: INFO: Container node-exporter ready: true, restart count 0 May 15 01:53:46.666: INFO: prometheus-operator-5bb8cb9d8f-fqb87 started at 2021-05-15 00:18:49 +0000 UTC (0+2 container statuses recorded) May 15 01:53:46.666: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:53:46.666: INFO: Container prometheus-operator ready: true, restart count 0 May 15 01:53:46.666: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.666: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:53:46.666: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.666: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:53:46.666: INFO: coredns-7677f9bb54-96w24 started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.666: INFO: Container coredns ready: true, restart count 0 May 15 01:53:46.666: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.666: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:53:46.666: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.666: INFO: Container kube-multus ready: true, restart count 1 W0515 01:53:46.676723 28 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:53:46.707: INFO: Latency metrics for node master2 May 15 01:53:46.707: INFO: Logging node info for node master3 May 15 01:53:46.710: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 162100 0 2021-05-14 19:57:14 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234767872 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324603904 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:53:45 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:53:45 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:53:45 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:53:45 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:53:46.710: INFO: Logging kubelet events for node master3 May 15 01:53:46.714: INFO: Logging pods the kubelet thinks is on node master3 May 15 01:53:46.723: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.723: INFO: Container kube-proxy ready: true, restart count 1 May 15 01:53:46.723: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:53:46.723: INFO: Init container install-cni ready: true, restart count 0 May 15 01:53:46.723: INFO: Container kube-flannel ready: true, restart count 2 May 15 01:53:46.723: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.723: INFO: Container kube-multus ready: true, restart count 1 May 15 01:53:46.723: INFO: coredns-7677f9bb54-rpj8c started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.723: INFO: Container coredns ready: true, restart count 0 May 15 01:53:46.723: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:53:46.723: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:53:46.723: INFO: Container node-exporter ready: true, restart count 0 May 15 01:53:46.723: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.723: INFO: Container kube-controller-manager ready: true, restart count 3 May 15 01:53:46.723: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.723: INFO: Container kube-scheduler ready: true, restart count 3 May 15 01:53:46.723: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.723: INFO: Container kube-apiserver ready: true, restart count 0 W0515 01:53:46.737646 28 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:53:46.771: INFO: Latency metrics for node master3 May 15 01:53:46.771: INFO: Logging node info for node node1 May 15 01:53:46.775: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 161975 0 2021-05-14 19:58:21 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1602":"csi-mock-csi-mock-volumes-1602","csi-mock-csi-mock-volumes-253":"csi-mock-csi-mock-volumes-253","csi-mock-csi-mock-volumes-2591":"csi-mock-csi-mock-volumes-2591","csi-mock-csi-mock-volumes-2993":"csi-mock-csi-mock-volumes-2993","csi-mock-csi-mock-volumes-6288":"csi-mock-csi-mock-volumes-6288","csi-mock-csi-mock-volumes-7734":"csi-mock-csi-mock-volumes-7734","csi-mock-csi-mock-volumes-895":"csi-mock-csi-mock-volumes-895","csi-mock-csi-mock-volumes-9232":"csi-mock-csi-mock-volumes-9232","csi-mock-csi-mock-volumes-9474":"csi-mock-csi-mock-volumes-9474","csi-mock-csi-mock-volumes-9728":"csi-mock-csi-mock-volumes-9728","csi-mock-csi-mock-volumes-9809":"csi-mock-csi-mock-volumes-9809"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:00 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:52:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}},"f:status":{"f:volumesAttached":{}}}} {kubelet Update v1 2021-05-15 01:52:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}},"f:volumesInUse":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:53:36 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:53:36 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:53:36 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:53:36 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[@ :],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-9232^4],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-9232^4,DevicePath:,},},Config:nil,},} May 15 01:53:46.776: INFO: Logging kubelet events for node node1 May 15 01:53:46.781: INFO: Logging pods the kubelet thinks is on node node1 May 15 01:53:46.796: INFO: hostexec-node1-c8b5r started at 2021-05-15 01:52:05 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.796: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:53:46.796: INFO: pod-83dd1c20-b22c-4938-b3e4-dcd2105d871b started at 2021-05-15 01:52:13 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.796: INFO: Container write-pod ready: false, restart count 0 May 15 01:53:46.796: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.796: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:53:46.796: INFO: node-feature-discovery-worker-bw8zg started at 2021-05-15 00:18:56 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.796: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:53:46.796: INFO: collectd-mrzps started at 2021-05-15 00:19:22 +0000 UTC (0+3 container statuses recorded) May 15 01:53:46.796: INFO: Container collectd ready: true, restart count 0 May 15 01:53:46.796: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:53:46.796: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:53:46.796: INFO: hostexec-node1-8nsmj started at 2021-05-15 01:53:10 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.796: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:53:46.796: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:53:46.796: INFO: Init container install-cni ready: true, restart count 0 May 15 01:53:46.796: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:53:46.796: INFO: prometheus-k8s-0 started at 2021-05-15 00:19:01 +0000 UTC (0+5 container statuses recorded) May 15 01:53:46.796: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 15 01:53:46.796: INFO: Container grafana ready: true, restart count 0 May 15 01:53:46.796: INFO: Container prometheus ready: true, restart count 26 May 15 01:53:46.796: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 15 01:53:46.796: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 15 01:53:46.796: INFO: pod-bdb45446-f731-4bb5-8170-cc86428a8ab2 started at 2021-05-15 01:53:29 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.796: INFO: Container write-pod ready: false, restart count 0 May 15 01:53:46.796: INFO: pod-aebbf8b7-c03c-4180-9a91-0c9b975a3efc started at 2021-05-15 01:52:59 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.796: INFO: Container write-pod ready: false, restart count 0 May 15 01:53:46.796: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.796: INFO: Container kube-multus ready: true, restart count 1 May 15 01:53:46.796: INFO: hostexec-node1-bdvv2 started at 2021-05-15 01:52:48 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.796: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:53:46.796: INFO: pod-bfba8786-cc42-4bcf-babe-8aa8c1a6f88f started at 2021-05-15 01:51:46 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.796: INFO: Container write-pod ready: false, restart count 0 May 15 01:53:46.796: INFO: csi-mockplugin-0 started at 2021-05-15 01:53:00 +0000 UTC (0+3 container statuses recorded) May 15 01:53:46.796: INFO: Container csi-provisioner ready: true, restart count 0 May 15 01:53:46.796: INFO: Container driver-registrar ready: true, restart count 0 May 15 01:53:46.796: INFO: Container mock ready: true, restart count 0 May 15 01:53:46.796: INFO: csi-mockplugin-attacher-0 started at 2021-05-15 01:53:00 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.796: INFO: Container csi-attacher ready: true, restart count 0 May 15 01:53:46.796: INFO: node-exporter-flvqz started at 2021-05-15 00:18:55 +0000 UTC (0+2 container statuses recorded) May 15 01:53:46.796: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:53:46.796: INFO: Container node-exporter ready: true, restart count 0 May 15 01:53:46.796: INFO: cmk-4s6dm started at 2021-05-15 00:18:54 +0000 UTC (0+2 container statuses recorded) May 15 01:53:46.796: INFO: Container nodereport ready: true, restart count 0 May 15 01:53:46.796: INFO: Container reconcile ready: true, restart count 0 May 15 01:53:46.796: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4vxcc started at 2021-05-15 00:19:00 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.796: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:53:46.796: INFO: hostexec-node1-vlzvd started at 2021-05-15 01:51:39 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.796: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:53:46.796: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.796: INFO: Container kube-proxy ready: true, restart count 2 W0515 01:53:46.809715 28 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:53:46.850: INFO: Latency metrics for node node1 May 15 01:53:46.850: INFO: Logging node info for node node2 May 15 01:53:46.853: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 162066 0 2021-05-14 19:58:22 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1663":"csi-mock-csi-mock-volumes-1663","csi-mock-csi-mock-volumes-3052":"csi-mock-csi-mock-volumes-3052","csi-mock-csi-mock-volumes-5200":"csi-mock-csi-mock-volumes-5200","csi-mock-csi-mock-volumes-5678":"csi-mock-csi-mock-volumes-5678","csi-mock-csi-mock-volumes-8760":"csi-mock-csi-mock-volumes-8760","csi-mock-csi-mock-volumes-9624":"csi-mock-csi-mock-volumes-9624"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:46 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:52:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:52:21 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:io.kubernetes.storage.mock/node":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:53:41 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:53:41 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:53:41 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:53:41 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:53:46.854: INFO: Logging kubelet events for node node2 May 15 01:53:46.858: INFO: Logging pods the kubelet thinks is on node node2 May 15 01:53:46.870: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.870: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:53:46.870: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 15 01:53:46.870: INFO: Container nodereport ready: true, restart count 0 May 15 01:53:46.870: INFO: Container reconcile ready: true, restart count 0 May 15 01:53:46.870: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.870: INFO: Container cmk-webhook ready: true, restart count 0 May 15 01:53:46.870: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 15 01:53:46.870: INFO: Container discover ready: false, restart count 0 May 15 01:53:46.870: INFO: Container init ready: false, restart count 0 May 15 01:53:46.870: INFO: Container install ready: false, restart count 0 May 15 01:53:46.870: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.870: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:53:46.870: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.870: INFO: Container kube-multus ready: true, restart count 1 May 15 01:53:46.870: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.870: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 15 01:53:46.870: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.871: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:53:46.871: INFO: configmap-client started at 2021-05-15 01:53:05 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.871: INFO: Container configmap-client ready: false, restart count 0 May 15 01:53:46.871: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:53:46.871: INFO: Init container install-cni ready: true, restart count 1 May 15 01:53:46.871: INFO: Container kube-flannel ready: true, restart count 4 May 15 01:53:46.871: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:53:46.871: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:53:46.871: INFO: Container node-exporter ready: true, restart count 0 May 15 01:53:46.871: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 15 01:53:46.871: INFO: Container tas-controller ready: true, restart count 0 May 15 01:53:46.871: INFO: Container tas-extender ready: true, restart count 0 May 15 01:53:46.871: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 15 01:53:46.871: INFO: Container collectd ready: true, restart count 0 May 15 01:53:46.871: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:53:46.871: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:53:46.871: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 15 01:53:46.871: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:53:46.871: INFO: pod-subpath-test-configmap-lp9m started at 2021-05-15 01:50:05 +0000 UTC (1+2 container statuses recorded) May 15 01:53:46.871: INFO: Init container init-volume-configmap-lp9m ready: false, restart count 0 May 15 01:53:46.871: INFO: Container test-container-subpath-configmap-lp9m ready: false, restart count 0 May 15 01:53:46.871: INFO: Container test-container-volume-configmap-lp9m ready: false, restart count 0 W0515 01:53:46.882870 28 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:53:46.943: INFO: Latency metrics for node node2 May 15 01:53:46.943: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-5959" for this suite. • Failure in Spec Setup (BeforeEach) [127.146 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: block] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 One pod requesting one prebound PVC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:205 should be able to mount volume and read from pod1 [BeforeEach] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:228 May 15 01:53:46.198: Unexpected error: <*errors.errorString | 0xc003cfcf00>: { s: "pod \"pod-bfba8786-cc42-4bcf-babe-8aa8c1a6f88f\" is not Running: timed out waiting for the condition", } pod "pod-bfba8786-cc42-4bcf-babe-8aa8c1a6f88f" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 ------------------------------ {"msg":"FAILED [sig-storage] PersistentVolumes-local [Volume type: block] One pod requesting one prebound PVC should be able to mount volume and read from pod1","total":-1,"completed":7,"skipped":825,"failed":3,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] One pod requesting one prebound PVC should be able to mount volume and read from pod1","[sig-storage] PersistentVolumes-local [Volume type: block] One pod requesting one prebound PVC should be able to mount volume and write from pod1","[sig-storage] PersistentVolumes-local [Volume type: block] One pod requesting one prebound PVC should be able to mount volume and read from pod1"]} May 15 01:53:46.957: INFO: Running AfterSuite actions on all nodes [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:52:05.523: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: dir-link-bindmounted] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes May 15 01:52:07.568: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir /tmp/local-volume-test-9cf5763a-3924-4f5f-878d-1d3ed574265f-backend && mount --bind /tmp/local-volume-test-9cf5763a-3924-4f5f-878d-1d3ed574265f-backend /tmp/local-volume-test-9cf5763a-3924-4f5f-878d-1d3ed574265f-backend && ln -s /tmp/local-volume-test-9cf5763a-3924-4f5f-878d-1d3ed574265f-backend /tmp/local-volume-test-9cf5763a-3924-4f5f-878d-1d3ed574265f] Namespace:persistent-local-volumes-test-90 PodName:hostexec-node1-c8b5r ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:52:07.568: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:52:07.679: INFO: Creating a PV followed by a PVC May 15 01:52:07.686: INFO: Waiting for PV local-pvnz8cw to bind to PVC pvc-qklw2 May 15 01:52:07.686: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-qklw2] to have phase Bound May 15 01:52:07.688: INFO: PersistentVolumeClaim pvc-qklw2 found but phase is Pending instead of Bound. May 15 01:52:09.691: INFO: PersistentVolumeClaim pvc-qklw2 found but phase is Pending instead of Bound. May 15 01:52:11.694: INFO: PersistentVolumeClaim pvc-qklw2 found but phase is Pending instead of Bound. May 15 01:52:13.696: INFO: PersistentVolumeClaim pvc-qklw2 found and phase=Bound (6.010655474s) May 15 01:52:13.696: INFO: Waiting up to 3m0s for PersistentVolume local-pvnz8cw to have phase Bound May 15 01:52:13.699: INFO: PersistentVolume local-pvnz8cw found and phase=Bound (2.86019ms) [It] should be able to write from pod1 and read from pod2 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:251 STEP: Creating pod1 STEP: Creating a pod May 15 01:54:13.724: FAIL: Unexpected error: <*errors.errorString | 0xc0011b26b0>: { s: "pod \"pod-83dd1c20-b22c-4938-b3e4-dcd2105d871b\" is not Running: timed out waiting for the condition", } pod "pod-83dd1c20-b22c-4938-b3e4-dcd2105d871b" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage.twoPodsReadWriteSerialTest(0xc002674580, 0xc0009ff300, 0xc001592300) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:764 +0xaf k8s.io/kubernetes/test/e2e/storage.glob..func20.2.5.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:252 +0x45 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc001d81980) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc001d81980) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc001d81980, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [Volume type: dir-link-bindmounted] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:54:13.726: INFO: Deleting PersistentVolumeClaim "pvc-qklw2" May 15 01:54:13.731: INFO: Deleting PersistentVolume "local-pvnz8cw" STEP: Removing the test directory May 15 01:54:13.736: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm /tmp/local-volume-test-9cf5763a-3924-4f5f-878d-1d3ed574265f && umount /tmp/local-volume-test-9cf5763a-3924-4f5f-878d-1d3ed574265f-backend && rm -r /tmp/local-volume-test-9cf5763a-3924-4f5f-878d-1d3ed574265f-backend] Namespace:persistent-local-volumes-test-90 PodName:hostexec-node1-c8b5r ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:54:13.736: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "persistent-local-volumes-test-90". STEP: Found 15 events. May 15 01:54:13.891: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for hostexec-node1-c8b5r: { } Scheduled: Successfully assigned persistent-local-volumes-test-90/hostexec-node1-c8b5r to node1 May 15 01:54:13.891: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-83dd1c20-b22c-4938-b3e4-dcd2105d871b: { } Scheduled: Successfully assigned persistent-local-volumes-test-90/pod-83dd1c20-b22c-4938-b3e4-dcd2105d871b to node1 May 15 01:54:13.891: INFO: At 2021-05-15 01:52:06 +0000 UTC - event for hostexec-node1-c8b5r: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 15 01:54:13.892: INFO: At 2021-05-15 01:52:07 +0000 UTC - event for hostexec-node1-c8b5r: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 491.014234ms May 15 01:54:13.892: INFO: At 2021-05-15 01:52:07 +0000 UTC - event for hostexec-node1-c8b5r: {kubelet node1} Created: Created container agnhost-container May 15 01:54:13.892: INFO: At 2021-05-15 01:52:07 +0000 UTC - event for hostexec-node1-c8b5r: {kubelet node1} Started: Started container agnhost-container May 15 01:54:13.892: INFO: At 2021-05-15 01:52:07 +0000 UTC - event for pvc-qklw2: {persistentvolume-controller } ProvisioningFailed: no volume plugin matched name: kubernetes.io/no-provisioner May 15 01:54:13.892: INFO: At 2021-05-15 01:52:15 +0000 UTC - event for pod-83dd1c20-b22c-4938-b3e4-dcd2105d871b: {multus } AddedInterface: Add eth0 [10.244.3.187/24] May 15 01:54:13.892: INFO: At 2021-05-15 01:52:15 +0000 UTC - event for pod-83dd1c20-b22c-4938-b3e4-dcd2105d871b: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:54:13.892: INFO: At 2021-05-15 01:52:16 +0000 UTC - event for pod-83dd1c20-b22c-4938-b3e4-dcd2105d871b: {kubelet node1} Failed: Error: ErrImagePull May 15 01:54:13.892: INFO: At 2021-05-15 01:52:16 +0000 UTC - event for pod-83dd1c20-b22c-4938-b3e4-dcd2105d871b: {kubelet node1} SandboxChanged: Pod sandbox changed, it will be killed and re-created. May 15 01:54:13.892: INFO: At 2021-05-15 01:52:16 +0000 UTC - event for pod-83dd1c20-b22c-4938-b3e4-dcd2105d871b: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:54:13.892: INFO: At 2021-05-15 01:52:18 +0000 UTC - event for pod-83dd1c20-b22c-4938-b3e4-dcd2105d871b: {multus } AddedInterface: Add eth0 [10.244.3.188/24] May 15 01:54:13.892: INFO: At 2021-05-15 01:52:18 +0000 UTC - event for pod-83dd1c20-b22c-4938-b3e4-dcd2105d871b: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 15 01:54:13.892: INFO: At 2021-05-15 01:52:18 +0000 UTC - event for pod-83dd1c20-b22c-4938-b3e4-dcd2105d871b: {kubelet node1} Failed: Error: ImagePullBackOff May 15 01:54:13.894: INFO: POD NODE PHASE GRACE CONDITIONS May 15 01:54:13.894: INFO: hostexec-node1-c8b5r node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:52:05 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:52:07 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:52:07 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:52:05 +0000 UTC }] May 15 01:54:13.894: INFO: pod-83dd1c20-b22c-4938-b3e4-dcd2105d871b node1 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:52:13 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:52:13 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:52:13 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:52:13 +0000 UTC }] May 15 01:54:13.894: INFO: May 15 01:54:13.898: INFO: Logging node info for node master1 May 15 01:54:13.900: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 162365 0 2021-05-14 19:56:35 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:09 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:09 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:09 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:54:09 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[@ :],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:54:13.901: INFO: Logging kubelet events for node master1 May 15 01:54:13.905: INFO: Logging pods the kubelet thinks is on node master1 May 15 01:54:13.913: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:54:13.913: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:54:13.913: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:54:13.913: INFO: Container kube-multus ready: true, restart count 1 May 15 01:54:13.913: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 15 01:54:13.913: INFO: Container docker-registry ready: true, restart count 0 May 15 01:54:13.913: INFO: Container nginx ready: true, restart count 0 May 15 01:54:13.913: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 15 01:54:13.913: INFO: Container nfd-controller ready: true, restart count 0 May 15 01:54:13.913: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:54:13.913: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:54:13.914: INFO: kubernetes-metrics-scraper-678c97765c-fswrn started at 2021-05-15 00:18:49 +0000 UTC (0+1 container statuses recorded) May 15 01:54:13.914: INFO: Container kubernetes-metrics-scraper ready: true, restart count 0 May 15 01:54:13.914: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:54:13.914: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:54:13.914: INFO: Container node-exporter ready: true, restart count 0 May 15 01:54:13.914: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:54:13.914: INFO: Container kube-scheduler ready: true, restart count 0 May 15 01:54:13.914: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:54:13.914: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:54:13.914: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:54:13.914: INFO: Init container install-cni ready: true, restart count 0 May 15 01:54:13.914: INFO: Container kube-flannel ready: true, restart count 1 W0515 01:54:13.926825 21 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:54:13.959: INFO: Latency metrics for node master1 May 15 01:54:13.959: INFO: Logging node info for node master2 May 15 01:54:13.961: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 162355 0 2021-05-14 19:57:04 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:07 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:07 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:07 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:54:07 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:54:13.962: INFO: Logging kubelet events for node master2 May 15 01:54:13.965: INFO: Logging pods the kubelet thinks is on node master2 May 15 01:54:13.975: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:54:13.975: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:54:13.976: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:54:13.976: INFO: Container kube-multus ready: true, restart count 1 May 15 01:54:13.976: INFO: coredns-7677f9bb54-96w24 started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:54:13.976: INFO: Container coredns ready: true, restart count 0 May 15 01:54:13.976: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:54:13.976: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:54:13.976: INFO: Container node-exporter ready: true, restart count 0 May 15 01:54:13.976: INFO: prometheus-operator-5bb8cb9d8f-fqb87 started at 2021-05-15 00:18:49 +0000 UTC (0+2 container statuses recorded) May 15 01:54:13.976: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:54:13.976: INFO: Container prometheus-operator ready: true, restart count 0 May 15 01:54:13.976: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:54:13.976: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:54:13.976: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:54:13.976: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:54:13.976: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:54:13.976: INFO: Container kube-scheduler ready: true, restart count 2 May 15 01:54:13.976: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:54:13.976: INFO: Init container install-cni ready: true, restart count 0 May 15 01:54:13.976: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:54:13.976: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 15 01:54:13.976: INFO: Container autoscaler ready: true, restart count 2 W0515 01:54:13.987513 21 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:54:14.011: INFO: Latency metrics for node master2 May 15 01:54:14.011: INFO: Logging node info for node master3 May 15 01:54:14.014: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 162345 0 2021-05-14 19:57:14 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234767872 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324603904 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:05 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:05 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:05 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:54:05 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:54:14.014: INFO: Logging kubelet events for node master3 May 15 01:54:14.019: INFO: Logging pods the kubelet thinks is on node master3 May 15 01:54:14.028: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:54:14.028: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:54:14.028: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:54:14.028: INFO: Container kube-proxy ready: true, restart count 1 May 15 01:54:14.028: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:54:14.028: INFO: Init container install-cni ready: true, restart count 0 May 15 01:54:14.028: INFO: Container kube-flannel ready: true, restart count 2 May 15 01:54:14.028: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:54:14.028: INFO: Container kube-multus ready: true, restart count 1 May 15 01:54:14.028: INFO: coredns-7677f9bb54-rpj8c started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:54:14.028: INFO: Container coredns ready: true, restart count 0 May 15 01:54:14.028: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:54:14.028: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:54:14.028: INFO: Container node-exporter ready: true, restart count 0 May 15 01:54:14.028: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 15 01:54:14.028: INFO: Container kube-controller-manager ready: true, restart count 3 May 15 01:54:14.028: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:54:14.028: INFO: Container kube-scheduler ready: true, restart count 3 W0515 01:54:14.040401 21 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:54:14.069: INFO: Latency metrics for node master3 May 15 01:54:14.069: INFO: Logging node info for node node1 May 15 01:54:14.072: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 162353 0 2021-05-14 19:58:21 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1602":"csi-mock-csi-mock-volumes-1602","csi-mock-csi-mock-volumes-253":"csi-mock-csi-mock-volumes-253","csi-mock-csi-mock-volumes-2591":"csi-mock-csi-mock-volumes-2591","csi-mock-csi-mock-volumes-2993":"csi-mock-csi-mock-volumes-2993","csi-mock-csi-mock-volumes-6288":"csi-mock-csi-mock-volumes-6288","csi-mock-csi-mock-volumes-7734":"csi-mock-csi-mock-volumes-7734","csi-mock-csi-mock-volumes-895":"csi-mock-csi-mock-volumes-895","csi-mock-csi-mock-volumes-9232":"csi-mock-csi-mock-volumes-9232","csi-mock-csi-mock-volumes-9474":"csi-mock-csi-mock-volumes-9474","csi-mock-csi-mock-volumes-9728":"csi-mock-csi-mock-volumes-9728","csi-mock-csi-mock-volumes-9809":"csi-mock-csi-mock-volumes-9809"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:00 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:52:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:53:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:07 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:07 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:07 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:54:07 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[@ :],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:54:14.073: INFO: Logging kubelet events for node node1 May 15 01:54:14.077: INFO: Logging pods the kubelet thinks is on node node1 May 15 01:54:14.089: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:54:14.089: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:54:14.089: INFO: hostexec-node1-c8b5r started at 2021-05-15 01:52:05 +0000 UTC (0+1 container statuses recorded) May 15 01:54:14.089: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:54:14.089: INFO: pod-83dd1c20-b22c-4938-b3e4-dcd2105d871b started at 2021-05-15 01:52:13 +0000 UTC (0+1 container statuses recorded) May 15 01:54:14.089: INFO: Container write-pod ready: false, restart count 0 May 15 01:54:14.089: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:54:14.089: INFO: Init container install-cni ready: true, restart count 0 May 15 01:54:14.089: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:54:14.089: INFO: node-feature-discovery-worker-bw8zg started at 2021-05-15 00:18:56 +0000 UTC (0+1 container statuses recorded) May 15 01:54:14.089: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:54:14.089: INFO: collectd-mrzps started at 2021-05-15 00:19:22 +0000 UTC (0+3 container statuses recorded) May 15 01:54:14.089: INFO: Container collectd ready: true, restart count 0 May 15 01:54:14.089: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:54:14.089: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:54:14.089: INFO: hostexec-node1-8nsmj started at 2021-05-15 01:53:10 +0000 UTC (0+1 container statuses recorded) May 15 01:54:14.089: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:54:14.089: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:54:14.089: INFO: Container kube-multus ready: true, restart count 1 May 15 01:54:14.089: INFO: prometheus-k8s-0 started at 2021-05-15 00:19:01 +0000 UTC (0+5 container statuses recorded) May 15 01:54:14.089: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 15 01:54:14.089: INFO: Container grafana ready: true, restart count 0 May 15 01:54:14.089: INFO: Container prometheus ready: true, restart count 26 May 15 01:54:14.089: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 15 01:54:14.089: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 15 01:54:14.089: INFO: pod-bdb45446-f731-4bb5-8170-cc86428a8ab2 started at 2021-05-15 01:53:29 +0000 UTC (0+1 container statuses recorded) May 15 01:54:14.089: INFO: Container write-pod ready: false, restart count 0 May 15 01:54:14.089: INFO: pod-aebbf8b7-c03c-4180-9a91-0c9b975a3efc started at 2021-05-15 01:52:59 +0000 UTC (0+1 container statuses recorded) May 15 01:54:14.089: INFO: Container write-pod ready: false, restart count 0 May 15 01:54:14.089: INFO: node-exporter-flvqz started at 2021-05-15 00:18:55 +0000 UTC (0+2 container statuses recorded) May 15 01:54:14.089: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:54:14.089: INFO: Container node-exporter ready: true, restart count 0 May 15 01:54:14.089: INFO: hostexec-node1-bdvv2 started at 2021-05-15 01:52:48 +0000 UTC (0+1 container statuses recorded) May 15 01:54:14.089: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:54:14.089: INFO: cmk-4s6dm started at 2021-05-15 00:18:54 +0000 UTC (0+2 container statuses recorded) May 15 01:54:14.089: INFO: Container nodereport ready: true, restart count 0 May 15 01:54:14.089: INFO: Container reconcile ready: true, restart count 0 May 15 01:54:14.089: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:54:14.089: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:54:14.089: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4vxcc started at 2021-05-15 00:19:00 +0000 UTC (0+1 container statuses recorded) May 15 01:54:14.089: INFO: Container kube-sriovdp ready: true, restart count 0 W0515 01:54:14.102436 21 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:54:14.158: INFO: Latency metrics for node node1 May 15 01:54:14.158: INFO: Logging node info for node node2 May 15 01:54:14.161: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 162375 0 2021-05-14 19:58:22 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1663":"csi-mock-csi-mock-volumes-1663","csi-mock-csi-mock-volumes-3052":"csi-mock-csi-mock-volumes-3052","csi-mock-csi-mock-volumes-5200":"csi-mock-csi-mock-volumes-5200","csi-mock-csi-mock-volumes-5678":"csi-mock-csi-mock-volumes-5678","csi-mock-csi-mock-volumes-8760":"csi-mock-csi-mock-volumes-8760","csi-mock-csi-mock-volumes-9624":"csi-mock-csi-mock-volumes-9624"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:46 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:52:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:52:21 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:io.kubernetes.storage.mock/node":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:11 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:11 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:11 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:54:11 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:54:14.161: INFO: Logging kubelet events for node node2 May 15 01:54:14.165: INFO: Logging pods the kubelet thinks is on node node2 May 15 01:54:14.175: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 15 01:54:14.175: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:54:14.175: INFO: pod-subpath-test-configmap-lp9m started at 2021-05-15 01:50:05 +0000 UTC (1+2 container statuses recorded) May 15 01:54:14.175: INFO: Init container init-volume-configmap-lp9m ready: false, restart count 0 May 15 01:54:14.175: INFO: Container test-container-subpath-configmap-lp9m ready: false, restart count 0 May 15 01:54:14.175: INFO: Container test-container-volume-configmap-lp9m ready: false, restart count 0 May 15 01:54:14.175: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 15 01:54:14.175: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:54:14.175: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 15 01:54:14.175: INFO: Container nodereport ready: true, restart count 0 May 15 01:54:14.175: INFO: Container reconcile ready: true, restart count 0 May 15 01:54:14.175: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 15 01:54:14.175: INFO: Container cmk-webhook ready: true, restart count 0 May 15 01:54:14.175: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 15 01:54:14.175: INFO: Container discover ready: false, restart count 0 May 15 01:54:14.175: INFO: Container init ready: false, restart count 0 May 15 01:54:14.175: INFO: Container install ready: false, restart count 0 May 15 01:54:14.175: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:54:14.175: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:54:14.175: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:54:14.175: INFO: Container kube-multus ready: true, restart count 1 May 15 01:54:14.175: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 15 01:54:14.175: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 15 01:54:14.175: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:54:14.175: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:54:14.175: INFO: configmap-client started at 2021-05-15 01:53:05 +0000 UTC (0+1 container statuses recorded) May 15 01:54:14.175: INFO: Container configmap-client ready: false, restart count 0 May 15 01:54:14.175: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:54:14.175: INFO: Init container install-cni ready: true, restart count 1 May 15 01:54:14.175: INFO: Container kube-flannel ready: true, restart count 4 May 15 01:54:14.175: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:54:14.175: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:54:14.175: INFO: Container node-exporter ready: true, restart count 0 May 15 01:54:14.175: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 15 01:54:14.175: INFO: Container tas-controller ready: true, restart count 0 May 15 01:54:14.175: INFO: Container tas-extender ready: true, restart count 0 May 15 01:54:14.175: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 15 01:54:14.175: INFO: Container collectd ready: true, restart count 0 May 15 01:54:14.175: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:54:14.175: INFO: Container rbac-proxy ready: true, restart count 0 W0515 01:54:14.184229 21 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:54:14.237: INFO: Latency metrics for node node2 May 15 01:54:14.237: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-90" for this suite. • Failure [128.721 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: dir-link-bindmounted] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 Two pods mounting a local volume one after the other /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:250 should be able to write from pod1 and read from pod2 [It] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:251 May 15 01:54:13.724: Unexpected error: <*errors.errorString | 0xc0011b26b0>: { s: "pod \"pod-83dd1c20-b22c-4938-b3e4-dcd2105d871b\" is not Running: timed out waiting for the condition", } pod "pod-83dd1c20-b22c-4938-b3e4-dcd2105d871b" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:764 ------------------------------ {"msg":"FAILED [sig-storage] PersistentVolumes-local [Volume type: dir-link-bindmounted] One pod requesting one prebound PVC should be able to mount volume and write from pod1","total":-1,"completed":4,"skipped":184,"failed":3,"failures":["[sig-storage] PersistentVolumes-local [Volume type: blockfswithoutformat] One pod requesting one prebound PVC should be able to mount volume and write from pod1","[sig-storage] PersistentVolumes-local [Volume type: block] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir-link-bindmounted] One pod requesting one prebound PVC should be able to mount volume and write from pod1"]} [BeforeEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:53:00.495: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename csi-mock-volumes STEP: Waiting for a default service account to be provisioned in namespace [It] should not be passed when podInfoOnMount=false /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:358 STEP: Building a driver namespace object, basename csi-mock-volumes-9232 STEP: Waiting for a default service account to be provisioned in namespace STEP: deploying csi mock driver May 15 01:53:00.563: INFO: creating *v1.ServiceAccount: csi-mock-volumes-9232-6013/csi-attacher May 15 01:53:00.566: INFO: creating *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-9232 May 15 01:53:00.566: INFO: Define cluster role external-attacher-runner-csi-mock-volumes-9232 May 15 01:53:00.568: INFO: creating *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-9232 May 15 01:53:00.571: INFO: creating *v1.Role: csi-mock-volumes-9232-6013/external-attacher-cfg-csi-mock-volumes-9232 May 15 01:53:00.574: INFO: creating *v1.RoleBinding: csi-mock-volumes-9232-6013/csi-attacher-role-cfg May 15 01:53:00.577: INFO: creating *v1.ServiceAccount: csi-mock-volumes-9232-6013/csi-provisioner May 15 01:53:00.579: INFO: creating *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-9232 May 15 01:53:00.579: INFO: Define cluster role external-provisioner-runner-csi-mock-volumes-9232 May 15 01:53:00.582: INFO: creating *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-9232 May 15 01:53:00.585: INFO: creating *v1.Role: csi-mock-volumes-9232-6013/external-provisioner-cfg-csi-mock-volumes-9232 May 15 01:53:00.588: INFO: creating *v1.RoleBinding: csi-mock-volumes-9232-6013/csi-provisioner-role-cfg May 15 01:53:00.590: INFO: creating *v1.ServiceAccount: csi-mock-volumes-9232-6013/csi-resizer May 15 01:53:00.594: INFO: creating *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-9232 May 15 01:53:00.594: INFO: Define cluster role external-resizer-runner-csi-mock-volumes-9232 May 15 01:53:00.597: INFO: creating *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-9232 May 15 01:53:00.599: INFO: creating *v1.Role: csi-mock-volumes-9232-6013/external-resizer-cfg-csi-mock-volumes-9232 May 15 01:53:00.602: INFO: creating *v1.RoleBinding: csi-mock-volumes-9232-6013/csi-resizer-role-cfg May 15 01:53:00.604: INFO: creating *v1.ServiceAccount: csi-mock-volumes-9232-6013/csi-mock May 15 01:53:00.607: INFO: creating *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-9232 May 15 01:53:00.609: INFO: creating *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-9232 May 15 01:53:00.613: INFO: creating *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-9232 May 15 01:53:00.615: INFO: creating *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-9232 May 15 01:53:00.618: INFO: creating *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-9232 May 15 01:53:00.621: INFO: creating *v1.StorageClass: csi-mock-sc-csi-mock-volumes-9232 May 15 01:53:00.624: INFO: creating *v1.StatefulSet: csi-mock-volumes-9232-6013/csi-mockplugin May 15 01:53:00.628: INFO: creating *v1.CSIDriver: csi-mock-csi-mock-volumes-9232 May 15 01:53:00.631: INFO: creating *v1.StatefulSet: csi-mock-volumes-9232-6013/csi-mockplugin-attacher May 15 01:53:00.634: INFO: waiting up to 4m0s for CSIDriver "csi-mock-csi-mock-volumes-9232" May 15 01:53:00.636: INFO: waiting for CSIDriver csi-mock-csi-mock-volumes-9232 to register on node node1 STEP: Creating pod May 15 01:53:16.907: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil May 15 01:53:16.913: INFO: Waiting up to 5m0s for PersistentVolumeClaims [pvc-8v88k] to have phase Bound May 15 01:53:16.916: INFO: PersistentVolumeClaim pvc-8v88k found but phase is Pending instead of Bound. May 15 01:53:18.921: INFO: PersistentVolumeClaim pvc-8v88k found and phase=Bound (2.007641872s) STEP: Deleting the previously created pod May 15 01:53:38.945: INFO: Deleting pod "pvc-volume-tester-5d5ns" in namespace "csi-mock-volumes-9232" May 15 01:53:38.949: INFO: Wait up to 5m0s for pod "pvc-volume-tester-5d5ns" to be fully deleted STEP: Checking CSI driver logs May 15 01:53:42.963: INFO: Found NodeUnpublishVolume: {json: {"Method":"/csi.v1.Node/NodeUnpublishVolume","Request":{"volume_id":"4","target_path":"/var/lib/kubelet/pods/2924650b-2871-4b19-a787-dcc2d145c7b3/volumes/kubernetes.io~csi/pvc-a252e0aa-15c8-4064-83e2-67bb7e56378a/mount"},"Response":{},"Error":"","FullError":null} Method:NodeUnpublishVolume Request:{VolumeContext:map[]} FullError:{Code:OK Message:} Error:} STEP: Deleting pod pvc-volume-tester-5d5ns May 15 01:53:42.963: INFO: Deleting pod "pvc-volume-tester-5d5ns" in namespace "csi-mock-volumes-9232" STEP: Deleting claim pvc-8v88k May 15 01:53:42.973: INFO: Waiting up to 2m0s for PersistentVolume pvc-a252e0aa-15c8-4064-83e2-67bb7e56378a to get deleted May 15 01:53:42.975: INFO: PersistentVolume pvc-a252e0aa-15c8-4064-83e2-67bb7e56378a found and phase=Bound (2.040203ms) May 15 01:53:44.978: INFO: PersistentVolume pvc-a252e0aa-15c8-4064-83e2-67bb7e56378a found and phase=Released (2.00592928s) May 15 01:53:46.981: INFO: PersistentVolume pvc-a252e0aa-15c8-4064-83e2-67bb7e56378a found and phase=Released (4.008335537s) May 15 01:53:48.984: INFO: PersistentVolume pvc-a252e0aa-15c8-4064-83e2-67bb7e56378a was removed STEP: Deleting storageclass csi-mock-volumes-9232-sc STEP: Cleaning up resources STEP: deleting the test namespace: csi-mock-volumes-9232 STEP: Waiting for namespaces [csi-mock-volumes-9232] to vanish STEP: uninstalling csi mock driver May 15 01:53:55.007: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-9232-6013/csi-attacher May 15 01:53:55.011: INFO: deleting *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-9232 May 15 01:53:55.017: INFO: deleting *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-9232 May 15 01:53:55.022: INFO: deleting *v1.Role: csi-mock-volumes-9232-6013/external-attacher-cfg-csi-mock-volumes-9232 May 15 01:53:55.026: INFO: deleting *v1.RoleBinding: csi-mock-volumes-9232-6013/csi-attacher-role-cfg May 15 01:53:55.031: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-9232-6013/csi-provisioner May 15 01:53:55.040: INFO: deleting *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-9232 May 15 01:53:55.050: INFO: deleting *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-9232 May 15 01:53:55.059: INFO: deleting *v1.Role: csi-mock-volumes-9232-6013/external-provisioner-cfg-csi-mock-volumes-9232 May 15 01:53:55.063: INFO: deleting *v1.RoleBinding: csi-mock-volumes-9232-6013/csi-provisioner-role-cfg May 15 01:53:55.067: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-9232-6013/csi-resizer May 15 01:53:55.071: INFO: deleting *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-9232 May 15 01:53:55.076: INFO: deleting *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-9232 May 15 01:53:55.080: INFO: deleting *v1.Role: csi-mock-volumes-9232-6013/external-resizer-cfg-csi-mock-volumes-9232 May 15 01:53:55.084: INFO: deleting *v1.RoleBinding: csi-mock-volumes-9232-6013/csi-resizer-role-cfg May 15 01:53:55.088: INFO: deleting *v1.ServiceAccount: csi-mock-volumes-9232-6013/csi-mock May 15 01:53:55.092: INFO: deleting *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-9232 May 15 01:53:55.096: INFO: deleting *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-9232 May 15 01:53:55.100: INFO: deleting *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-9232 May 15 01:53:55.104: INFO: deleting *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-9232 May 15 01:53:55.108: INFO: deleting *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-9232 May 15 01:53:55.112: INFO: deleting *v1.StorageClass: csi-mock-sc-csi-mock-volumes-9232 May 15 01:53:55.116: INFO: deleting *v1.StatefulSet: csi-mock-volumes-9232-6013/csi-mockplugin May 15 01:53:55.121: INFO: deleting *v1.CSIDriver: csi-mock-csi-mock-volumes-9232 May 15 01:53:55.125: INFO: deleting *v1.StatefulSet: csi-mock-volumes-9232-6013/csi-mockplugin-attacher STEP: deleting the driver namespace: csi-mock-volumes-9232-6013 STEP: Waiting for namespaces [csi-mock-volumes-9232-6013] to vanish [AfterEach] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 May 15 01:54:23.139: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "csi-mock-volumes-9232" for this suite. May 15 01:54:23.147: INFO: Namespace csi-mock-volumes-9232 was already deleted STEP: Destroying namespace "csi-mock-volumes-9232-6013" for this suite. May 15 01:54:23.151: INFO: Namespace csi-mock-volumes-9232-6013 was already deleted • [SLOW TEST:82.656 seconds] [sig-storage] CSI mock volume /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 CSI workload information using mock driver /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:308 should not be passed when podInfoOnMount=false /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/csi_mock_volume.go:358 ------------------------------ {"msg":"PASSED [sig-storage] CSI mock volume CSI workload information using mock driver should not be passed when podInfoOnMount=false","total":-1,"completed":5,"skipped":184,"failed":3,"failures":["[sig-storage] PersistentVolumes-local [Volume type: blockfswithoutformat] One pod requesting one prebound PVC should be able to mount volume and write from pod1","[sig-storage] PersistentVolumes-local [Volume type: block] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir-link-bindmounted] One pod requesting one prebound PVC should be able to mount volume and write from pod1"]} May 15 01:54:23.153: INFO: Running AfterSuite actions on all nodes [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:52:48.566: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: blockfswithformat] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes STEP: Creating block device on node "node1" using path "/tmp/local-volume-test-93c55bfc-3336-424c-8718-2e4cf5d2a7b6" May 15 01:52:54.630: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-93c55bfc-3336-424c-8718-2e4cf5d2a7b6 && dd if=/dev/zero of=/tmp/local-volume-test-93c55bfc-3336-424c-8718-2e4cf5d2a7b6/file bs=4096 count=5120 && losetup -f /tmp/local-volume-test-93c55bfc-3336-424c-8718-2e4cf5d2a7b6/file] Namespace:persistent-local-volumes-test-3852 PodName:hostexec-node1-bdvv2 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:52:54.630: INFO: >>> kubeConfig: /root/.kube/config May 15 01:52:54.768: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c E2E_LOOP_DEV=$(losetup | grep /tmp/local-volume-test-93c55bfc-3336-424c-8718-2e4cf5d2a7b6/file | awk '{ print $1 }') 2>&1 > /dev/null && echo ${E2E_LOOP_DEV}] Namespace:persistent-local-volumes-test-3852 PodName:hostexec-node1-bdvv2 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:52:54.768: INFO: >>> kubeConfig: /root/.kube/config May 15 01:52:54.880: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkfs -t ext4 /dev/loop2 && mount -t ext4 /dev/loop2 /tmp/local-volume-test-93c55bfc-3336-424c-8718-2e4cf5d2a7b6 && chmod o+rwx /tmp/local-volume-test-93c55bfc-3336-424c-8718-2e4cf5d2a7b6] Namespace:persistent-local-volumes-test-3852 PodName:hostexec-node1-bdvv2 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:52:54.880: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:52:55.168: INFO: Creating a PV followed by a PVC May 15 01:52:55.176: INFO: Waiting for PV local-pv5pgbf to bind to PVC pvc-cm74r May 15 01:52:55.176: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-cm74r] to have phase Bound May 15 01:52:55.178: INFO: PersistentVolumeClaim pvc-cm74r found but phase is Pending instead of Bound. May 15 01:52:57.182: INFO: PersistentVolumeClaim pvc-cm74r found but phase is Pending instead of Bound. May 15 01:52:59.186: INFO: PersistentVolumeClaim pvc-cm74r found and phase=Bound (4.010252299s) May 15 01:52:59.186: INFO: Waiting up to 3m0s for PersistentVolume local-pv5pgbf to have phase Bound May 15 01:52:59.188: INFO: PersistentVolume local-pv5pgbf found and phase=Bound (2.110151ms) [It] should be able to write from pod1 and read from pod2 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:245 STEP: Creating pod1 to write to the PV STEP: Creating a pod May 15 01:54:59.213: FAIL: Unexpected error: <*errors.errorString | 0xc0029f22b0>: { s: "pod \"pod-aebbf8b7-c03c-4180-9a91-0c9b975a3efc\" is not Running: timed out waiting for the condition", } pod "pod-aebbf8b7-c03c-4180-9a91-0c9b975a3efc" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage.twoPodsReadWriteTest(0xc0028351e0, 0xc0015b3380, 0xc00257faa0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:727 +0xaf k8s.io/kubernetes/test/e2e/storage.glob..func20.2.4.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:246 +0x45 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc00362b080) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc00362b080) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc00362b080, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [Volume type: blockfswithformat] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:54:59.215: INFO: Deleting PersistentVolumeClaim "pvc-cm74r" May 15 01:54:59.221: INFO: Deleting PersistentVolume "local-pv5pgbf" May 15 01:54:59.226: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c umount /tmp/local-volume-test-93c55bfc-3336-424c-8718-2e4cf5d2a7b6] Namespace:persistent-local-volumes-test-3852 PodName:hostexec-node1-bdvv2 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:54:59.226: INFO: >>> kubeConfig: /root/.kube/config May 15 01:54:59.364: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c E2E_LOOP_DEV=$(losetup | grep /tmp/local-volume-test-93c55bfc-3336-424c-8718-2e4cf5d2a7b6/file | awk '{ print $1 }') 2>&1 > /dev/null && echo ${E2E_LOOP_DEV}] Namespace:persistent-local-volumes-test-3852 PodName:hostexec-node1-bdvv2 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:54:59.364: INFO: >>> kubeConfig: /root/.kube/config STEP: Tear down block device "/dev/loop2" on node "node1" at path /tmp/local-volume-test-93c55bfc-3336-424c-8718-2e4cf5d2a7b6/file May 15 01:54:59.483: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c losetup -d /dev/loop2] Namespace:persistent-local-volumes-test-3852 PodName:hostexec-node1-bdvv2 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:54:59.483: INFO: >>> kubeConfig: /root/.kube/config STEP: Removing the test directory /tmp/local-volume-test-93c55bfc-3336-424c-8718-2e4cf5d2a7b6 May 15 01:54:59.604: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-93c55bfc-3336-424c-8718-2e4cf5d2a7b6] Namespace:persistent-local-volumes-test-3852 PodName:hostexec-node1-bdvv2 ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:54:59.604: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "persistent-local-volumes-test-3852". STEP: Found 15 events. May 15 01:54:59.726: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for hostexec-node1-bdvv2: { } Scheduled: Successfully assigned persistent-local-volumes-test-3852/hostexec-node1-bdvv2 to node1 May 15 01:54:59.726: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-aebbf8b7-c03c-4180-9a91-0c9b975a3efc: { } Scheduled: Successfully assigned persistent-local-volumes-test-3852/pod-aebbf8b7-c03c-4180-9a91-0c9b975a3efc to node1 May 15 01:54:59.726: INFO: At 2021-05-15 01:52:49 +0000 UTC - event for hostexec-node1-bdvv2: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 15 01:54:59.726: INFO: At 2021-05-15 01:52:49 +0000 UTC - event for hostexec-node1-bdvv2: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 438.355926ms May 15 01:54:59.726: INFO: At 2021-05-15 01:52:50 +0000 UTC - event for hostexec-node1-bdvv2: {kubelet node1} Created: Created container agnhost-container May 15 01:54:59.726: INFO: At 2021-05-15 01:52:50 +0000 UTC - event for hostexec-node1-bdvv2: {kubelet node1} Started: Started container agnhost-container May 15 01:54:59.726: INFO: At 2021-05-15 01:52:55 +0000 UTC - event for pvc-cm74r: {persistentvolume-controller } ProvisioningFailed: no volume plugin matched name: kubernetes.io/no-provisioner May 15 01:54:59.726: INFO: At 2021-05-15 01:53:01 +0000 UTC - event for pod-aebbf8b7-c03c-4180-9a91-0c9b975a3efc: {multus } AddedInterface: Add eth0 [10.244.3.199/24] May 15 01:54:59.726: INFO: At 2021-05-15 01:53:01 +0000 UTC - event for pod-aebbf8b7-c03c-4180-9a91-0c9b975a3efc: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:54:59.726: INFO: At 2021-05-15 01:53:02 +0000 UTC - event for pod-aebbf8b7-c03c-4180-9a91-0c9b975a3efc: {kubelet node1} Failed: Error: ErrImagePull May 15 01:54:59.726: INFO: At 2021-05-15 01:53:02 +0000 UTC - event for pod-aebbf8b7-c03c-4180-9a91-0c9b975a3efc: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:54:59.726: INFO: At 2021-05-15 01:53:03 +0000 UTC - event for pod-aebbf8b7-c03c-4180-9a91-0c9b975a3efc: {kubelet node1} SandboxChanged: Pod sandbox changed, it will be killed and re-created. May 15 01:54:59.726: INFO: At 2021-05-15 01:53:06 +0000 UTC - event for pod-aebbf8b7-c03c-4180-9a91-0c9b975a3efc: {multus } AddedInterface: Add eth0 [10.244.3.202/24] May 15 01:54:59.726: INFO: At 2021-05-15 01:53:06 +0000 UTC - event for pod-aebbf8b7-c03c-4180-9a91-0c9b975a3efc: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 15 01:54:59.726: INFO: At 2021-05-15 01:53:06 +0000 UTC - event for pod-aebbf8b7-c03c-4180-9a91-0c9b975a3efc: {kubelet node1} Failed: Error: ImagePullBackOff May 15 01:54:59.728: INFO: POD NODE PHASE GRACE CONDITIONS May 15 01:54:59.728: INFO: hostexec-node1-bdvv2 node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:52:48 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:52:50 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:52:50 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:52:48 +0000 UTC }] May 15 01:54:59.728: INFO: pod-aebbf8b7-c03c-4180-9a91-0c9b975a3efc node1 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:52:59 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:52:59 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:52:59 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:52:59 +0000 UTC }] May 15 01:54:59.728: INFO: May 15 01:54:59.734: INFO: Logging node info for node master1 May 15 01:54:59.736: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 162603 0 2021-05-14 19:56:35 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:59 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:59 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:59 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:54:59 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[@ :],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:54:59.736: INFO: Logging kubelet events for node master1 May 15 01:54:59.741: INFO: Logging pods the kubelet thinks is on node master1 May 15 01:54:59.751: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 15 01:54:59.751: INFO: Container docker-registry ready: true, restart count 0 May 15 01:54:59.751: INFO: Container nginx ready: true, restart count 0 May 15 01:54:59.751: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 15 01:54:59.751: INFO: Container nfd-controller ready: true, restart count 0 May 15 01:54:59.751: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:54:59.751: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:54:59.751: INFO: kubernetes-metrics-scraper-678c97765c-fswrn started at 2021-05-15 00:18:49 +0000 UTC (0+1 container statuses recorded) May 15 01:54:59.751: INFO: Container kubernetes-metrics-scraper ready: true, restart count 0 May 15 01:54:59.751: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:54:59.751: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:54:59.751: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:54:59.751: INFO: Container kube-multus ready: true, restart count 1 May 15 01:54:59.751: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:54:59.751: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:54:59.751: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:54:59.751: INFO: Init container install-cni ready: true, restart count 0 May 15 01:54:59.751: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:54:59.751: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:54:59.751: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:54:59.751: INFO: Container node-exporter ready: true, restart count 0 May 15 01:54:59.751: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:54:59.751: INFO: Container kube-scheduler ready: true, restart count 0 W0515 01:54:59.763431 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:54:59.797: INFO: Latency metrics for node master1 May 15 01:54:59.797: INFO: Logging node info for node master2 May 15 01:54:59.799: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 162593 0 2021-05-14 19:57:04 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:57 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:57 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:57 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:54:57 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:54:59.800: INFO: Logging kubelet events for node master2 May 15 01:54:59.804: INFO: Logging pods the kubelet thinks is on node master2 May 15 01:54:59.813: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:54:59.813: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:54:59.813: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:54:59.813: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:54:59.813: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:54:59.813: INFO: Container kube-scheduler ready: true, restart count 2 May 15 01:54:59.813: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:54:59.813: INFO: Init container install-cni ready: true, restart count 0 May 15 01:54:59.813: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:54:59.813: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 15 01:54:59.813: INFO: Container autoscaler ready: true, restart count 2 May 15 01:54:59.813: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:54:59.813: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:54:59.813: INFO: Container node-exporter ready: true, restart count 0 May 15 01:54:59.813: INFO: prometheus-operator-5bb8cb9d8f-fqb87 started at 2021-05-15 00:18:49 +0000 UTC (0+2 container statuses recorded) May 15 01:54:59.813: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:54:59.813: INFO: Container prometheus-operator ready: true, restart count 0 May 15 01:54:59.813: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:54:59.813: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:54:59.813: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:54:59.813: INFO: Container kube-multus ready: true, restart count 1 May 15 01:54:59.813: INFO: coredns-7677f9bb54-96w24 started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:54:59.813: INFO: Container coredns ready: true, restart count 0 W0515 01:54:59.827853 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:54:59.854: INFO: Latency metrics for node master2 May 15 01:54:59.854: INFO: Logging node info for node master3 May 15 01:54:59.857: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 162584 0 2021-05-14 19:57:14 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234767872 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324603904 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:55 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:55 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:55 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:54:55 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:54:59.858: INFO: Logging kubelet events for node master3 May 15 01:54:59.864: INFO: Logging pods the kubelet thinks is on node master3 May 15 01:54:59.872: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:54:59.872: INFO: Init container install-cni ready: true, restart count 0 May 15 01:54:59.872: INFO: Container kube-flannel ready: true, restart count 2 May 15 01:54:59.872: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:54:59.872: INFO: Container kube-multus ready: true, restart count 1 May 15 01:54:59.872: INFO: coredns-7677f9bb54-rpj8c started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:54:59.872: INFO: Container coredns ready: true, restart count 0 May 15 01:54:59.872: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:54:59.872: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:54:59.872: INFO: Container node-exporter ready: true, restart count 0 May 15 01:54:59.872: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 15 01:54:59.872: INFO: Container kube-controller-manager ready: true, restart count 3 May 15 01:54:59.872: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:54:59.872: INFO: Container kube-scheduler ready: true, restart count 3 May 15 01:54:59.872: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:54:59.872: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:54:59.872: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:54:59.872: INFO: Container kube-proxy ready: true, restart count 1 W0515 01:54:59.886812 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:54:59.911: INFO: Latency metrics for node master3 May 15 01:54:59.911: INFO: Logging node info for node node1 May 15 01:54:59.914: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 162592 0 2021-05-14 19:58:21 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1602":"csi-mock-csi-mock-volumes-1602","csi-mock-csi-mock-volumes-253":"csi-mock-csi-mock-volumes-253","csi-mock-csi-mock-volumes-2591":"csi-mock-csi-mock-volumes-2591","csi-mock-csi-mock-volumes-2993":"csi-mock-csi-mock-volumes-2993","csi-mock-csi-mock-volumes-6288":"csi-mock-csi-mock-volumes-6288","csi-mock-csi-mock-volumes-7734":"csi-mock-csi-mock-volumes-7734","csi-mock-csi-mock-volumes-895":"csi-mock-csi-mock-volumes-895","csi-mock-csi-mock-volumes-9232":"csi-mock-csi-mock-volumes-9232","csi-mock-csi-mock-volumes-9474":"csi-mock-csi-mock-volumes-9474","csi-mock-csi-mock-volumes-9728":"csi-mock-csi-mock-volumes-9728","csi-mock-csi-mock-volumes-9809":"csi-mock-csi-mock-volumes-9809"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:00 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:52:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:53:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:57 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:57 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:57 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:54:57 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[@ :],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:54:59.915: INFO: Logging kubelet events for node node1 May 15 01:54:59.919: INFO: Logging pods the kubelet thinks is on node node1 May 15 01:54:59.934: INFO: cmk-4s6dm started at 2021-05-15 00:18:54 +0000 UTC (0+2 container statuses recorded) May 15 01:54:59.934: INFO: Container nodereport ready: true, restart count 0 May 15 01:54:59.934: INFO: Container reconcile ready: true, restart count 0 May 15 01:54:59.934: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4vxcc started at 2021-05-15 00:19:00 +0000 UTC (0+1 container statuses recorded) May 15 01:54:59.934: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:54:59.934: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:54:59.934: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:54:59.934: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:54:59.934: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:54:59.934: INFO: node-feature-discovery-worker-bw8zg started at 2021-05-15 00:18:56 +0000 UTC (0+1 container statuses recorded) May 15 01:54:59.934: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:54:59.934: INFO: collectd-mrzps started at 2021-05-15 00:19:22 +0000 UTC (0+3 container statuses recorded) May 15 01:54:59.934: INFO: Container collectd ready: true, restart count 0 May 15 01:54:59.934: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:54:59.934: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:54:59.934: INFO: hostexec-node1-8nsmj started at 2021-05-15 01:53:10 +0000 UTC (0+1 container statuses recorded) May 15 01:54:59.934: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:54:59.934: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:54:59.934: INFO: Init container install-cni ready: true, restart count 0 May 15 01:54:59.934: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:54:59.934: INFO: prometheus-k8s-0 started at 2021-05-15 00:19:01 +0000 UTC (0+5 container statuses recorded) May 15 01:54:59.934: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 15 01:54:59.934: INFO: Container grafana ready: true, restart count 0 May 15 01:54:59.934: INFO: Container prometheus ready: true, restart count 26 May 15 01:54:59.934: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 15 01:54:59.934: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 15 01:54:59.934: INFO: pod-bdb45446-f731-4bb5-8170-cc86428a8ab2 started at 2021-05-15 01:53:29 +0000 UTC (0+1 container statuses recorded) May 15 01:54:59.934: INFO: Container write-pod ready: false, restart count 0 May 15 01:54:59.934: INFO: pod-aebbf8b7-c03c-4180-9a91-0c9b975a3efc started at 2021-05-15 01:52:59 +0000 UTC (0+1 container statuses recorded) May 15 01:54:59.934: INFO: Container write-pod ready: false, restart count 0 May 15 01:54:59.934: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:54:59.934: INFO: Container kube-multus ready: true, restart count 1 May 15 01:54:59.934: INFO: hostexec-node1-bdvv2 started at 2021-05-15 01:52:48 +0000 UTC (0+1 container statuses recorded) May 15 01:54:59.934: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:54:59.934: INFO: node-exporter-flvqz started at 2021-05-15 00:18:55 +0000 UTC (0+2 container statuses recorded) May 15 01:54:59.934: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:54:59.934: INFO: Container node-exporter ready: true, restart count 0 W0515 01:54:59.946988 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:54:59.991: INFO: Latency metrics for node node1 May 15 01:54:59.991: INFO: Logging node info for node node2 May 15 01:54:59.993: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 162573 0 2021-05-14 19:58:22 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1663":"csi-mock-csi-mock-volumes-1663","csi-mock-csi-mock-volumes-3052":"csi-mock-csi-mock-volumes-3052","csi-mock-csi-mock-volumes-5200":"csi-mock-csi-mock-volumes-5200","csi-mock-csi-mock-volumes-5678":"csi-mock-csi-mock-volumes-5678","csi-mock-csi-mock-volumes-8760":"csi-mock-csi-mock-volumes-8760","csi-mock-csi-mock-volumes-9624":"csi-mock-csi-mock-volumes-9624"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:46 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:52:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:52:21 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:io.kubernetes.storage.mock/node":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:52 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:52 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:52 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:54:52 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:54:59.993: INFO: Logging kubelet events for node node2 May 15 01:54:59.997: INFO: Logging pods the kubelet thinks is on node node2 May 15 01:55:00.009: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:55:00.009: INFO: Init container install-cni ready: true, restart count 1 May 15 01:55:00.009: INFO: Container kube-flannel ready: true, restart count 4 May 15 01:55:00.009: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:55:00.009: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:55:00.009: INFO: Container node-exporter ready: true, restart count 0 May 15 01:55:00.009: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 15 01:55:00.009: INFO: Container tas-controller ready: true, restart count 0 May 15 01:55:00.009: INFO: Container tas-extender ready: true, restart count 0 May 15 01:55:00.009: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 15 01:55:00.009: INFO: Container collectd ready: true, restart count 0 May 15 01:55:00.009: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:55:00.009: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:55:00.009: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 15 01:55:00.009: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:55:00.009: INFO: pod-subpath-test-configmap-lp9m started at 2021-05-15 01:50:05 +0000 UTC (1+2 container statuses recorded) May 15 01:55:00.009: INFO: Init container init-volume-configmap-lp9m ready: false, restart count 0 May 15 01:55:00.009: INFO: Container test-container-subpath-configmap-lp9m ready: false, restart count 0 May 15 01:55:00.009: INFO: Container test-container-volume-configmap-lp9m ready: false, restart count 0 May 15 01:55:00.009: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 15 01:55:00.009: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:55:00.009: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 15 01:55:00.009: INFO: Container nodereport ready: true, restart count 0 May 15 01:55:00.009: INFO: Container reconcile ready: true, restart count 0 May 15 01:55:00.009: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 15 01:55:00.009: INFO: Container cmk-webhook ready: true, restart count 0 May 15 01:55:00.009: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 15 01:55:00.009: INFO: Container discover ready: false, restart count 0 May 15 01:55:00.009: INFO: Container init ready: false, restart count 0 May 15 01:55:00.009: INFO: Container install ready: false, restart count 0 May 15 01:55:00.009: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 15 01:55:00.009: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 15 01:55:00.009: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:55:00.009: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:55:00.009: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:55:00.009: INFO: Container kube-multus ready: true, restart count 1 May 15 01:55:00.009: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:55:00.009: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:55:00.009: INFO: configmap-client started at 2021-05-15 01:53:05 +0000 UTC (0+1 container statuses recorded) May 15 01:55:00.009: INFO: Container configmap-client ready: false, restart count 0 W0515 01:55:00.022635 39 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:55:00.058: INFO: Latency metrics for node node2 May 15 01:55:00.058: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-3852" for this suite. • Failure [131.501 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: blockfswithformat] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 Two pods mounting a local volume at the same time /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:244 should be able to write from pod1 and read from pod2 [It] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:245 May 15 01:54:59.213: Unexpected error: <*errors.errorString | 0xc0029f22b0>: { s: "pod \"pod-aebbf8b7-c03c-4180-9a91-0c9b975a3efc\" is not Running: timed out waiting for the condition", } pod "pod-aebbf8b7-c03c-4180-9a91-0c9b975a3efc" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:727 ------------------------------ {"msg":"FAILED [sig-storage] PersistentVolumes-local [Volume type: blockfswithformat] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2","total":-1,"completed":6,"skipped":989,"failed":3,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] One pod requesting one prebound PVC should be able to mount volume and write from pod1","[sig-storage] PersistentVolumes-local [Volume type: blockfswithformat] One pod requesting one prebound PVC should be able to mount volume and write from pod1","[sig-storage] PersistentVolumes-local [Volume type: blockfswithformat] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2"]} May 15 01:55:00.078: INFO: Running AfterSuite actions on all nodes [BeforeEach] [sig-storage] Subpath /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:50:05.054: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename subpath STEP: Waiting for a default service account to be provisioned in namespace [It] should verify that container can restart successfully after configmaps modified /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/subpath.go:131 STEP: Create configmap STEP: Creating pod pod-subpath-test-configmap-lp9m May 15 01:55:05.097: FAIL: while waiting for pod to be running Unexpected error: <*errors.errorString | 0xc0003021f0>: { s: "timed out waiting for the condition", } timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.testPodContainerRestartWithHooks(0xc001fc5e40, 0xc00136c000, 0xc0016e5018) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:807 +0x54c k8s.io/kubernetes/test/e2e/storage/testsuites.TestPodContainerRestartWithConfigmapModified(0xc001fc5e40, 0xc001ffca00, 0xc001ffcb40) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:924 +0x498 k8s.io/kubernetes/test/e2e/storage.glob..func25.2.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/subpath.go:134 +0x177 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc002e7a600) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc002e7a600) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc002e7a600, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 May 15 01:55:05.098: INFO: Deleting pod "pod-subpath-test-configmap-lp9m" in namespace "subpath-4323" May 15 01:55:05.103: INFO: Wait up to 5m0s for pod "pod-subpath-test-configmap-lp9m" to be fully deleted [AfterEach] [sig-storage] Subpath /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "subpath-4323". STEP: Found 9 events. May 15 01:55:07.112: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-subpath-test-configmap-lp9m: { } Scheduled: Successfully assigned subpath-4323/pod-subpath-test-configmap-lp9m to node2 May 15 01:55:07.112: INFO: At 2021-05-15 01:50:06 +0000 UTC - event for pod-subpath-test-configmap-lp9m: {multus } AddedInterface: Add eth0 [10.244.4.228/24] May 15 01:55:07.112: INFO: At 2021-05-15 01:50:06 +0000 UTC - event for pod-subpath-test-configmap-lp9m: {kubelet node2} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:55:07.112: INFO: At 2021-05-15 01:50:07 +0000 UTC - event for pod-subpath-test-configmap-lp9m: {kubelet node2} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:55:07.112: INFO: At 2021-05-15 01:50:07 +0000 UTC - event for pod-subpath-test-configmap-lp9m: {kubelet node2} Failed: Error: ErrImagePull May 15 01:55:07.112: INFO: At 2021-05-15 01:50:07 +0000 UTC - event for pod-subpath-test-configmap-lp9m: {kubelet node2} SandboxChanged: Pod sandbox changed, it will be killed and re-created. May 15 01:55:07.112: INFO: At 2021-05-15 01:50:09 +0000 UTC - event for pod-subpath-test-configmap-lp9m: {multus } AddedInterface: Add eth0 [10.244.4.229/24] May 15 01:55:07.112: INFO: At 2021-05-15 01:50:09 +0000 UTC - event for pod-subpath-test-configmap-lp9m: {kubelet node2} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 15 01:55:07.112: INFO: At 2021-05-15 01:50:09 +0000 UTC - event for pod-subpath-test-configmap-lp9m: {kubelet node2} Failed: Error: ImagePullBackOff May 15 01:55:07.115: INFO: POD NODE PHASE GRACE CONDITIONS May 15 01:55:07.115: INFO: May 15 01:55:07.119: INFO: Logging node info for node master1 May 15 01:55:07.122: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 162603 0 2021-05-14 19:56:35 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:59 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:59 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:59 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:54:59 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[@ :],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:55:07.122: INFO: Logging kubelet events for node master1 May 15 01:55:07.127: INFO: Logging pods the kubelet thinks is on node master1 May 15 01:55:07.136: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.136: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:55:07.136: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:55:07.136: INFO: Init container install-cni ready: true, restart count 0 May 15 01:55:07.136: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:55:07.136: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:55:07.136: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:55:07.136: INFO: Container node-exporter ready: true, restart count 0 May 15 01:55:07.136: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.136: INFO: Container kube-scheduler ready: true, restart count 0 May 15 01:55:07.136: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 15 01:55:07.136: INFO: Container docker-registry ready: true, restart count 0 May 15 01:55:07.136: INFO: Container nginx ready: true, restart count 0 May 15 01:55:07.136: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.136: INFO: Container nfd-controller ready: true, restart count 0 May 15 01:55:07.136: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.136: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:55:07.136: INFO: kubernetes-metrics-scraper-678c97765c-fswrn started at 2021-05-15 00:18:49 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.136: INFO: Container kubernetes-metrics-scraper ready: true, restart count 0 May 15 01:55:07.136: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.136: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:55:07.136: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.136: INFO: Container kube-multus ready: true, restart count 1 W0515 01:55:07.150190 29 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:55:07.182: INFO: Latency metrics for node master1 May 15 01:55:07.182: INFO: Logging node info for node master2 May 15 01:55:07.185: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 162593 0 2021-05-14 19:57:04 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:57 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:57 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:57 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:54:57 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:55:07.185: INFO: Logging kubelet events for node master2 May 15 01:55:07.189: INFO: Logging pods the kubelet thinks is on node master2 May 15 01:55:07.198: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.198: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:55:07.198: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.198: INFO: Container kube-multus ready: true, restart count 1 May 15 01:55:07.198: INFO: coredns-7677f9bb54-96w24 started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.198: INFO: Container coredns ready: true, restart count 0 May 15 01:55:07.198: INFO: prometheus-operator-5bb8cb9d8f-fqb87 started at 2021-05-15 00:18:49 +0000 UTC (0+2 container statuses recorded) May 15 01:55:07.198: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:55:07.198: INFO: Container prometheus-operator ready: true, restart count 0 May 15 01:55:07.198: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.198: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:55:07.198: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.198: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:55:07.198: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.198: INFO: Container kube-scheduler ready: true, restart count 2 May 15 01:55:07.198: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:55:07.198: INFO: Init container install-cni ready: true, restart count 0 May 15 01:55:07.198: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:55:07.198: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.198: INFO: Container autoscaler ready: true, restart count 2 May 15 01:55:07.198: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:55:07.198: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:55:07.198: INFO: Container node-exporter ready: true, restart count 0 W0515 01:55:07.211662 29 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:55:07.242: INFO: Latency metrics for node master2 May 15 01:55:07.242: INFO: Logging node info for node master3 May 15 01:55:07.244: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 162655 0 2021-05-14 19:57:14 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234767872 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324603904 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:55:05 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:55:05 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:55:05 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:55:05 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:55:07.245: INFO: Logging kubelet events for node master3 May 15 01:55:07.250: INFO: Logging pods the kubelet thinks is on node master3 May 15 01:55:07.258: INFO: coredns-7677f9bb54-rpj8c started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.258: INFO: Container coredns ready: true, restart count 0 May 15 01:55:07.258: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:55:07.258: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:55:07.258: INFO: Container node-exporter ready: true, restart count 0 May 15 01:55:07.258: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.258: INFO: Container kube-controller-manager ready: true, restart count 3 May 15 01:55:07.258: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.258: INFO: Container kube-scheduler ready: true, restart count 3 May 15 01:55:07.258: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.258: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:55:07.258: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.258: INFO: Container kube-proxy ready: true, restart count 1 May 15 01:55:07.258: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:55:07.258: INFO: Init container install-cni ready: true, restart count 0 May 15 01:55:07.258: INFO: Container kube-flannel ready: true, restart count 2 May 15 01:55:07.258: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.258: INFO: Container kube-multus ready: true, restart count 1 W0515 01:55:07.271051 29 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:55:07.302: INFO: Latency metrics for node master3 May 15 01:55:07.302: INFO: Logging node info for node node1 May 15 01:55:07.305: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 162592 0 2021-05-14 19:58:21 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1602":"csi-mock-csi-mock-volumes-1602","csi-mock-csi-mock-volumes-253":"csi-mock-csi-mock-volumes-253","csi-mock-csi-mock-volumes-2591":"csi-mock-csi-mock-volumes-2591","csi-mock-csi-mock-volumes-2993":"csi-mock-csi-mock-volumes-2993","csi-mock-csi-mock-volumes-6288":"csi-mock-csi-mock-volumes-6288","csi-mock-csi-mock-volumes-7734":"csi-mock-csi-mock-volumes-7734","csi-mock-csi-mock-volumes-895":"csi-mock-csi-mock-volumes-895","csi-mock-csi-mock-volumes-9232":"csi-mock-csi-mock-volumes-9232","csi-mock-csi-mock-volumes-9474":"csi-mock-csi-mock-volumes-9474","csi-mock-csi-mock-volumes-9728":"csi-mock-csi-mock-volumes-9728","csi-mock-csi-mock-volumes-9809":"csi-mock-csi-mock-volumes-9809"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:00 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:52:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:53:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:57 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:57 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:54:57 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:54:57 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[@ :],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:55:07.305: INFO: Logging kubelet events for node node1 May 15 01:55:07.311: INFO: Logging pods the kubelet thinks is on node node1 May 15 01:55:07.322: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.322: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:55:07.322: INFO: node-feature-discovery-worker-bw8zg started at 2021-05-15 00:18:56 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.322: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:55:07.322: INFO: collectd-mrzps started at 2021-05-15 00:19:22 +0000 UTC (0+3 container statuses recorded) May 15 01:55:07.322: INFO: Container collectd ready: true, restart count 0 May 15 01:55:07.322: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:55:07.322: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:55:07.322: INFO: hostexec-node1-8nsmj started at 2021-05-15 01:53:10 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.322: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:55:07.322: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:55:07.322: INFO: Init container install-cni ready: true, restart count 0 May 15 01:55:07.322: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:55:07.322: INFO: prometheus-k8s-0 started at 2021-05-15 00:19:01 +0000 UTC (0+5 container statuses recorded) May 15 01:55:07.322: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 15 01:55:07.322: INFO: Container grafana ready: true, restart count 0 May 15 01:55:07.322: INFO: Container prometheus ready: true, restart count 26 May 15 01:55:07.322: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 15 01:55:07.322: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 15 01:55:07.322: INFO: pod-bdb45446-f731-4bb5-8170-cc86428a8ab2 started at 2021-05-15 01:53:29 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.322: INFO: Container write-pod ready: false, restart count 0 May 15 01:55:07.322: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.322: INFO: Container kube-multus ready: true, restart count 1 May 15 01:55:07.322: INFO: node-exporter-flvqz started at 2021-05-15 00:18:55 +0000 UTC (0+2 container statuses recorded) May 15 01:55:07.322: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:55:07.322: INFO: Container node-exporter ready: true, restart count 0 May 15 01:55:07.322: INFO: cmk-4s6dm started at 2021-05-15 00:18:54 +0000 UTC (0+2 container statuses recorded) May 15 01:55:07.322: INFO: Container nodereport ready: true, restart count 0 May 15 01:55:07.322: INFO: Container reconcile ready: true, restart count 0 May 15 01:55:07.322: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4vxcc started at 2021-05-15 00:19:00 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.322: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:55:07.322: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.323: INFO: Container kube-proxy ready: true, restart count 2 W0515 01:55:07.334887 29 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:55:07.376: INFO: Latency metrics for node node1 May 15 01:55:07.376: INFO: Logging node info for node node2 May 15 01:55:07.379: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 162619 0 2021-05-14 19:58:22 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1663":"csi-mock-csi-mock-volumes-1663","csi-mock-csi-mock-volumes-3052":"csi-mock-csi-mock-volumes-3052","csi-mock-csi-mock-volumes-5200":"csi-mock-csi-mock-volumes-5200","csi-mock-csi-mock-volumes-5678":"csi-mock-csi-mock-volumes-5678","csi-mock-csi-mock-volumes-8760":"csi-mock-csi-mock-volumes-8760","csi-mock-csi-mock-volumes-9624":"csi-mock-csi-mock-volumes-9624"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:46 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:52:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:52:21 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:io.kubernetes.storage.mock/node":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:55:02 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:55:02 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:55:02 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:55:02 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:55:07.380: INFO: Logging kubelet events for node node2 May 15 01:55:07.384: INFO: Logging pods the kubelet thinks is on node node2 May 15 01:55:07.395: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.395: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:55:07.395: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.395: INFO: Container kube-multus ready: true, restart count 1 May 15 01:55:07.395: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.395: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 15 01:55:07.395: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.395: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:55:07.395: INFO: configmap-client started at 2021-05-15 01:53:05 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.395: INFO: Container configmap-client ready: false, restart count 0 May 15 01:55:07.395: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:55:07.395: INFO: Init container install-cni ready: true, restart count 1 May 15 01:55:07.395: INFO: Container kube-flannel ready: true, restart count 4 May 15 01:55:07.395: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:55:07.395: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:55:07.395: INFO: Container node-exporter ready: true, restart count 0 May 15 01:55:07.395: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 15 01:55:07.395: INFO: Container tas-controller ready: true, restart count 0 May 15 01:55:07.395: INFO: Container tas-extender ready: true, restart count 0 May 15 01:55:07.395: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 15 01:55:07.395: INFO: Container collectd ready: true, restart count 0 May 15 01:55:07.395: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:55:07.395: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:55:07.395: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.395: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:55:07.395: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.395: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:55:07.395: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 15 01:55:07.395: INFO: Container nodereport ready: true, restart count 0 May 15 01:55:07.395: INFO: Container reconcile ready: true, restart count 0 May 15 01:55:07.395: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 15 01:55:07.395: INFO: Container cmk-webhook ready: true, restart count 0 May 15 01:55:07.395: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 15 01:55:07.395: INFO: Container discover ready: false, restart count 0 May 15 01:55:07.395: INFO: Container init ready: false, restart count 0 May 15 01:55:07.395: INFO: Container install ready: false, restart count 0 W0515 01:55:07.408934 29 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:55:07.456: INFO: Latency metrics for node node2 May 15 01:55:07.456: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "subpath-4323" for this suite. • Failure [302.412 seconds] [sig-storage] Subpath /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 Container restart /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/subpath.go:130 should verify that container can restart successfully after configmaps modified [It] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/subpath.go:131 May 15 01:55:05.097: while waiting for pod to be running Unexpected error: <*errors.errorString | 0xc0003021f0>: { s: "timed out waiting for the condition", } timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:807 ------------------------------ {"msg":"FAILED [sig-storage] Subpath Container restart should verify that container can restart successfully after configmaps modified","total":-1,"completed":5,"skipped":531,"failed":2,"failures":["[sig-storage] PersistentVolumes-local [Volume type: blockfswithformat] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2","[sig-storage] Subpath Container restart should verify that container can restart successfully after configmaps modified"]} May 15 01:55:07.471: INFO: Running AfterSuite actions on all nodes [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:53:10.656: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:155 [BeforeEach] [Volume type: block] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:191 STEP: Initializing test volumes STEP: Creating block device on node "node1" using path "/tmp/local-volume-test-6471d326-1a27-41cc-b881-eab7687c3aa3" May 15 01:53:18.702: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-6471d326-1a27-41cc-b881-eab7687c3aa3 && dd if=/dev/zero of=/tmp/local-volume-test-6471d326-1a27-41cc-b881-eab7687c3aa3/file bs=4096 count=5120 && losetup -f /tmp/local-volume-test-6471d326-1a27-41cc-b881-eab7687c3aa3/file] Namespace:persistent-local-volumes-test-4115 PodName:hostexec-node1-8nsmj ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:53:18.702: INFO: >>> kubeConfig: /root/.kube/config May 15 01:53:18.880: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c E2E_LOOP_DEV=$(losetup | grep /tmp/local-volume-test-6471d326-1a27-41cc-b881-eab7687c3aa3/file | awk '{ print $1 }') 2>&1 > /dev/null && echo ${E2E_LOOP_DEV}] Namespace:persistent-local-volumes-test-4115 PodName:hostexec-node1-8nsmj ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:53:18.880: INFO: >>> kubeConfig: /root/.kube/config STEP: Creating local PVCs and PVs May 15 01:53:19.144: INFO: Creating a PV followed by a PVC May 15 01:53:19.150: INFO: Waiting for PV local-pvqv5sl to bind to PVC pvc-ztslz May 15 01:53:19.150: INFO: Waiting up to 3m0s for PersistentVolumeClaims [pvc-ztslz] to have phase Bound May 15 01:53:19.152: INFO: PersistentVolumeClaim pvc-ztslz found but phase is Pending instead of Bound. May 15 01:53:21.158: INFO: PersistentVolumeClaim pvc-ztslz found but phase is Pending instead of Bound. May 15 01:53:23.161: INFO: PersistentVolumeClaim pvc-ztslz found but phase is Pending instead of Bound. May 15 01:53:25.166: INFO: PersistentVolumeClaim pvc-ztslz found but phase is Pending instead of Bound. May 15 01:53:27.169: INFO: PersistentVolumeClaim pvc-ztslz found but phase is Pending instead of Bound. May 15 01:53:29.174: INFO: PersistentVolumeClaim pvc-ztslz found and phase=Bound (10.024582911s) May 15 01:53:29.174: INFO: Waiting up to 3m0s for PersistentVolume local-pvqv5sl to have phase Bound May 15 01:53:29.176: INFO: PersistentVolume local-pvqv5sl found and phase=Bound (1.880846ms) [It] should be able to write from pod1 and read from pod2 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:245 STEP: Creating pod1 to write to the PV STEP: Creating a pod May 15 01:55:29.199: FAIL: Unexpected error: <*errors.errorString | 0xc00257aa50>: { s: "pod \"pod-bdb45446-f731-4bb5-8170-cc86428a8ab2\" is not Running: timed out waiting for the condition", } pod "pod-bdb45446-f731-4bb5-8170-cc86428a8ab2" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage.twoPodsReadWriteTest(0xc002b7eb00, 0xc00390f480, 0xc001a92660) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:727 +0xaf k8s.io/kubernetes/test/e2e/storage.glob..func20.2.4.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:246 +0x45 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc001881380) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc001881380) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc001881380, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [Volume type: block] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:200 STEP: Cleaning up PVC and PV May 15 01:55:29.201: INFO: Deleting PersistentVolumeClaim "pvc-ztslz" May 15 01:55:29.208: INFO: Deleting PersistentVolume "local-pvqv5sl" May 15 01:55:29.212: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c E2E_LOOP_DEV=$(losetup | grep /tmp/local-volume-test-6471d326-1a27-41cc-b881-eab7687c3aa3/file | awk '{ print $1 }') 2>&1 > /dev/null && echo ${E2E_LOOP_DEV}] Namespace:persistent-local-volumes-test-4115 PodName:hostexec-node1-8nsmj ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:55:29.212: INFO: >>> kubeConfig: /root/.kube/config STEP: Tear down block device "/dev/loop3" on node "node1" at path /tmp/local-volume-test-6471d326-1a27-41cc-b881-eab7687c3aa3/file May 15 01:55:29.352: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c losetup -d /dev/loop3] Namespace:persistent-local-volumes-test-4115 PodName:hostexec-node1-8nsmj ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:55:29.352: INFO: >>> kubeConfig: /root/.kube/config STEP: Removing the test directory /tmp/local-volume-test-6471d326-1a27-41cc-b881-eab7687c3aa3 May 15 01:55:29.463: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-volume-test-6471d326-1a27-41cc-b881-eab7687c3aa3] Namespace:persistent-local-volumes-test-4115 PodName:hostexec-node1-8nsmj ContainerName:agnhost-container Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:true} May 15 01:55:29.463: INFO: >>> kubeConfig: /root/.kube/config [AfterEach] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "persistent-local-volumes-test-4115". STEP: Found 18 events. May 15 01:55:29.603: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for hostexec-node1-8nsmj: { } Scheduled: Successfully assigned persistent-local-volumes-test-4115/hostexec-node1-8nsmj to node1 May 15 01:55:29.603: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-bdb45446-f731-4bb5-8170-cc86428a8ab2: { } Scheduled: Successfully assigned persistent-local-volumes-test-4115/pod-bdb45446-f731-4bb5-8170-cc86428a8ab2 to node1 May 15 01:55:29.603: INFO: At 2021-05-15 01:53:13 +0000 UTC - event for hostexec-node1-8nsmj: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 15 01:55:29.603: INFO: At 2021-05-15 01:53:13 +0000 UTC - event for hostexec-node1-8nsmj: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 453.033927ms May 15 01:55:29.603: INFO: At 2021-05-15 01:53:13 +0000 UTC - event for hostexec-node1-8nsmj: {kubelet node1} Created: Created container agnhost-container May 15 01:55:29.603: INFO: At 2021-05-15 01:53:13 +0000 UTC - event for hostexec-node1-8nsmj: {kubelet node1} Started: Started container agnhost-container May 15 01:55:29.603: INFO: At 2021-05-15 01:53:19 +0000 UTC - event for pvc-ztslz: {persistentvolume-controller } ProvisioningFailed: no volume plugin matched name: kubernetes.io/no-provisioner May 15 01:55:29.603: INFO: At 2021-05-15 01:53:30 +0000 UTC - event for pod-bdb45446-f731-4bb5-8170-cc86428a8ab2: {kubelet node1} SuccessfulMountVolume: MapVolume.MapPodDevice succeeded for volume "local-pvqv5sl" volumeMapPath "/var/lib/kubelet/pods/d16d6f52-5109-4a60-9e1e-bc5dc0433171/volumeDevices/kubernetes.io~local-volume" May 15 01:55:29.603: INFO: At 2021-05-15 01:53:30 +0000 UTC - event for pod-bdb45446-f731-4bb5-8170-cc86428a8ab2: {kubelet node1} SuccessfulMountVolume: MapVolume.MapPodDevice succeeded for volume "local-pvqv5sl" globalMapPath "/var/lib/kubelet/plugins/kubernetes.io~local-volume/volumeDevices/local-pvqv5sl" May 15 01:55:29.603: INFO: At 2021-05-15 01:53:31 +0000 UTC - event for pod-bdb45446-f731-4bb5-8170-cc86428a8ab2: {multus } AddedInterface: Add eth0 [10.244.3.205/24] May 15 01:55:29.603: INFO: At 2021-05-15 01:53:31 +0000 UTC - event for pod-bdb45446-f731-4bb5-8170-cc86428a8ab2: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:55:29.603: INFO: At 2021-05-15 01:53:32 +0000 UTC - event for pod-bdb45446-f731-4bb5-8170-cc86428a8ab2: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:55:29.603: INFO: At 2021-05-15 01:53:32 +0000 UTC - event for pod-bdb45446-f731-4bb5-8170-cc86428a8ab2: {kubelet node1} Failed: Error: ErrImagePull May 15 01:55:29.603: INFO: At 2021-05-15 01:53:33 +0000 UTC - event for pod-bdb45446-f731-4bb5-8170-cc86428a8ab2: {kubelet node1} SandboxChanged: Pod sandbox changed, it will be killed and re-created. May 15 01:55:29.603: INFO: At 2021-05-15 01:53:35 +0000 UTC - event for pod-bdb45446-f731-4bb5-8170-cc86428a8ab2: {multus } AddedInterface: Add eth0 [10.244.3.206/24] May 15 01:55:29.603: INFO: At 2021-05-15 01:53:35 +0000 UTC - event for pod-bdb45446-f731-4bb5-8170-cc86428a8ab2: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 15 01:55:29.603: INFO: At 2021-05-15 01:53:35 +0000 UTC - event for pod-bdb45446-f731-4bb5-8170-cc86428a8ab2: {kubelet node1} Failed: Error: ImagePullBackOff May 15 01:55:29.603: INFO: At 2021-05-15 01:53:52 +0000 UTC - event for pod-bdb45446-f731-4bb5-8170-cc86428a8ab2: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:55:29.606: INFO: POD NODE PHASE GRACE CONDITIONS May 15 01:55:29.606: INFO: hostexec-node1-8nsmj node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:53:10 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:53:14 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:53:14 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:53:10 +0000 UTC }] May 15 01:55:29.606: INFO: pod-bdb45446-f731-4bb5-8170-cc86428a8ab2 node1 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:53:29 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:53:29 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:53:29 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-15 01:53:29 +0000 UTC }] May 15 01:55:29.606: INFO: May 15 01:55:29.610: INFO: Logging node info for node master1 May 15 01:55:29.612: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 162723 0 2021-05-14 19:56:35 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:55:19 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:55:19 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:55:19 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:55:19 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[@ :],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:55:29.613: INFO: Logging kubelet events for node master1 May 15 01:55:29.617: INFO: Logging pods the kubelet thinks is on node master1 May 15 01:55:29.626: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.626: INFO: Container kube-scheduler ready: true, restart count 0 May 15 01:55:29.626: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.626: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:55:29.626: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:55:29.626: INFO: Init container install-cni ready: true, restart count 0 May 15 01:55:29.626: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:55:29.626: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:55:29.626: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:55:29.626: INFO: Container node-exporter ready: true, restart count 0 May 15 01:55:29.626: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.626: INFO: Container kube-multus ready: true, restart count 1 May 15 01:55:29.627: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 15 01:55:29.627: INFO: Container docker-registry ready: true, restart count 0 May 15 01:55:29.627: INFO: Container nginx ready: true, restart count 0 May 15 01:55:29.627: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.627: INFO: Container nfd-controller ready: true, restart count 0 May 15 01:55:29.627: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.627: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:55:29.627: INFO: kubernetes-metrics-scraper-678c97765c-fswrn started at 2021-05-15 00:18:49 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.627: INFO: Container kubernetes-metrics-scraper ready: true, restart count 0 May 15 01:55:29.627: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.627: INFO: Container kube-proxy ready: true, restart count 2 W0515 01:55:29.640366 24 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:55:29.674: INFO: Latency metrics for node master1 May 15 01:55:29.674: INFO: Logging node info for node master2 May 15 01:55:29.676: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 162751 0 2021-05-14 19:57:04 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:55:27 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:55:27 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:55:27 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:55:27 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:55:29.677: INFO: Logging kubelet events for node master2 May 15 01:55:29.682: INFO: Logging pods the kubelet thinks is on node master2 May 15 01:55:29.692: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.692: INFO: Container autoscaler ready: true, restart count 2 May 15 01:55:29.692: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:55:29.692: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:55:29.692: INFO: Container node-exporter ready: true, restart count 0 May 15 01:55:29.692: INFO: prometheus-operator-5bb8cb9d8f-fqb87 started at 2021-05-15 00:18:49 +0000 UTC (0+2 container statuses recorded) May 15 01:55:29.692: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:55:29.692: INFO: Container prometheus-operator ready: true, restart count 0 May 15 01:55:29.692: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.692: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:55:29.692: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.692: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:55:29.692: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.692: INFO: Container kube-scheduler ready: true, restart count 2 May 15 01:55:29.692: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:55:29.692: INFO: Init container install-cni ready: true, restart count 0 May 15 01:55:29.692: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:55:29.692: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.692: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:55:29.692: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.692: INFO: Container kube-multus ready: true, restart count 1 May 15 01:55:29.692: INFO: coredns-7677f9bb54-96w24 started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.692: INFO: Container coredns ready: true, restart count 0 W0515 01:55:29.704987 24 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:55:29.729: INFO: Latency metrics for node master2 May 15 01:55:29.729: INFO: Logging node info for node master3 May 15 01:55:29.732: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 162743 0 2021-05-14 19:57:14 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234767872 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324603904 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:55:25 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:55:25 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:55:25 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:55:25 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:55:29.732: INFO: Logging kubelet events for node master3 May 15 01:55:29.737: INFO: Logging pods the kubelet thinks is on node master3 May 15 01:55:29.743: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:55:29.744: INFO: Init container install-cni ready: true, restart count 0 May 15 01:55:29.744: INFO: Container kube-flannel ready: true, restart count 2 May 15 01:55:29.744: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.744: INFO: Container kube-multus ready: true, restart count 1 May 15 01:55:29.744: INFO: coredns-7677f9bb54-rpj8c started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.744: INFO: Container coredns ready: true, restart count 0 May 15 01:55:29.744: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:55:29.744: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:55:29.744: INFO: Container node-exporter ready: true, restart count 0 May 15 01:55:29.744: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.744: INFO: Container kube-controller-manager ready: true, restart count 3 May 15 01:55:29.744: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.744: INFO: Container kube-scheduler ready: true, restart count 3 May 15 01:55:29.744: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.744: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:55:29.744: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.744: INFO: Container kube-proxy ready: true, restart count 1 W0515 01:55:29.757309 24 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:55:29.786: INFO: Latency metrics for node master3 May 15 01:55:29.786: INFO: Logging node info for node node1 May 15 01:55:29.788: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 162750 0 2021-05-14 19:58:21 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1602":"csi-mock-csi-mock-volumes-1602","csi-mock-csi-mock-volumes-253":"csi-mock-csi-mock-volumes-253","csi-mock-csi-mock-volumes-2591":"csi-mock-csi-mock-volumes-2591","csi-mock-csi-mock-volumes-2993":"csi-mock-csi-mock-volumes-2993","csi-mock-csi-mock-volumes-6288":"csi-mock-csi-mock-volumes-6288","csi-mock-csi-mock-volumes-7734":"csi-mock-csi-mock-volumes-7734","csi-mock-csi-mock-volumes-895":"csi-mock-csi-mock-volumes-895","csi-mock-csi-mock-volumes-9232":"csi-mock-csi-mock-volumes-9232","csi-mock-csi-mock-volumes-9474":"csi-mock-csi-mock-volumes-9474","csi-mock-csi-mock-volumes-9728":"csi-mock-csi-mock-volumes-9728","csi-mock-csi-mock-volumes-9809":"csi-mock-csi-mock-volumes-9809"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:00 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:52:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:53:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:55:27 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:55:27 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:55:27 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:55:27 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[@ :],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:55:29.789: INFO: Logging kubelet events for node node1 May 15 01:55:29.793: INFO: Logging pods the kubelet thinks is on node node1 May 15 01:55:29.804: INFO: cmk-4s6dm started at 2021-05-15 00:18:54 +0000 UTC (0+2 container statuses recorded) May 15 01:55:29.804: INFO: Container nodereport ready: true, restart count 0 May 15 01:55:29.804: INFO: Container reconcile ready: true, restart count 0 May 15 01:55:29.804: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.804: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:55:29.804: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4vxcc started at 2021-05-15 00:19:00 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.804: INFO: Container kube-sriovdp ready: true, restart count 0 May 15 01:55:29.804: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.804: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:55:29.804: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:55:29.804: INFO: Init container install-cni ready: true, restart count 0 May 15 01:55:29.804: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:55:29.804: INFO: node-feature-discovery-worker-bw8zg started at 2021-05-15 00:18:56 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.804: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:55:29.804: INFO: collectd-mrzps started at 2021-05-15 00:19:22 +0000 UTC (0+3 container statuses recorded) May 15 01:55:29.804: INFO: Container collectd ready: true, restart count 0 May 15 01:55:29.804: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:55:29.804: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:55:29.804: INFO: hostexec-node1-8nsmj started at 2021-05-15 01:53:10 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.804: INFO: Container agnhost-container ready: true, restart count 0 May 15 01:55:29.804: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.804: INFO: Container kube-multus ready: true, restart count 1 May 15 01:55:29.804: INFO: prometheus-k8s-0 started at 2021-05-15 00:19:01 +0000 UTC (0+5 container statuses recorded) May 15 01:55:29.804: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 15 01:55:29.804: INFO: Container grafana ready: true, restart count 0 May 15 01:55:29.804: INFO: Container prometheus ready: true, restart count 26 May 15 01:55:29.804: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 15 01:55:29.804: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 15 01:55:29.804: INFO: pod-bdb45446-f731-4bb5-8170-cc86428a8ab2 started at 2021-05-15 01:53:29 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.804: INFO: Container write-pod ready: false, restart count 0 May 15 01:55:29.804: INFO: node-exporter-flvqz started at 2021-05-15 00:18:55 +0000 UTC (0+2 container statuses recorded) May 15 01:55:29.804: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:55:29.804: INFO: Container node-exporter ready: true, restart count 0 W0515 01:55:29.817387 24 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:55:29.859: INFO: Latency metrics for node node1 May 15 01:55:29.859: INFO: Logging node info for node node2 May 15 01:55:29.862: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 162734 0 2021-05-14 19:58:22 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1663":"csi-mock-csi-mock-volumes-1663","csi-mock-csi-mock-volumes-3052":"csi-mock-csi-mock-volumes-3052","csi-mock-csi-mock-volumes-5200":"csi-mock-csi-mock-volumes-5200","csi-mock-csi-mock-volumes-5678":"csi-mock-csi-mock-volumes-5678","csi-mock-csi-mock-volumes-8760":"csi-mock-csi-mock-volumes-8760","csi-mock-csi-mock-volumes-9624":"csi-mock-csi-mock-volumes-9624"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:46 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:52:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:52:21 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:io.kubernetes.storage.mock/node":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:55:22 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:55:22 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:55:22 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:55:22 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:55:29.863: INFO: Logging kubelet events for node node2 May 15 01:55:29.868: INFO: Logging pods the kubelet thinks is on node node2 May 15 01:55:29.879: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.879: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:55:29.880: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 15 01:55:29.880: INFO: Container nodereport ready: true, restart count 0 May 15 01:55:29.880: INFO: Container reconcile ready: true, restart count 0 May 15 01:55:29.880: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.880: INFO: Container cmk-webhook ready: true, restart count 0 May 15 01:55:29.880: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 15 01:55:29.880: INFO: Container discover ready: false, restart count 0 May 15 01:55:29.880: INFO: Container init ready: false, restart count 0 May 15 01:55:29.880: INFO: Container install ready: false, restart count 0 May 15 01:55:29.880: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.880: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:55:29.880: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.880: INFO: Container kube-multus ready: true, restart count 1 May 15 01:55:29.880: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.880: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 15 01:55:29.880: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.880: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:55:29.880: INFO: configmap-client started at 2021-05-15 01:53:05 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.880: INFO: Container configmap-client ready: false, restart count 0 May 15 01:55:29.880: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:55:29.880: INFO: Init container install-cni ready: true, restart count 1 May 15 01:55:29.880: INFO: Container kube-flannel ready: true, restart count 4 May 15 01:55:29.880: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:55:29.880: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:55:29.880: INFO: Container node-exporter ready: true, restart count 0 May 15 01:55:29.880: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 15 01:55:29.880: INFO: Container tas-controller ready: true, restart count 0 May 15 01:55:29.880: INFO: Container tas-extender ready: true, restart count 0 May 15 01:55:29.880: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 15 01:55:29.880: INFO: Container collectd ready: true, restart count 0 May 15 01:55:29.880: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:55:29.880: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:55:29.880: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 15 01:55:29.880: INFO: Container kube-sriovdp ready: true, restart count 0 W0515 01:55:29.891700 24 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:55:29.938: INFO: Latency metrics for node node2 May 15 01:55:29.938: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "persistent-local-volumes-test-4115" for this suite. • Failure [139.290 seconds] [sig-storage] PersistentVolumes-local /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 [Volume type: block] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:188 Two pods mounting a local volume at the same time /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:244 should be able to write from pod1 and read from pod2 [It] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:245 May 15 01:55:29.199: Unexpected error: <*errors.errorString | 0xc00257aa50>: { s: "pod \"pod-bdb45446-f731-4bb5-8170-cc86428a8ab2\" is not Running: timed out waiting for the condition", } pod "pod-bdb45446-f731-4bb5-8170-cc86428a8ab2" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:727 ------------------------------ {"msg":"FAILED [sig-storage] PersistentVolumes-local [Volume type: block] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2","total":-1,"completed":8,"skipped":1148,"failed":4,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir] One pod requesting one prebound PVC should be able to mount volume and write from pod1","[sig-storage] PersistentVolumes-local [Volume type: tmpfs] One pod requesting one prebound PVC should be able to mount volume and read from pod1","[sig-storage] PersistentVolumes-local [Volume type: dir-link-bindmounted] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: block] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2"]} May 15 01:55:29.951: INFO: Running AfterSuite actions on all nodes [BeforeEach] [sig-storage] Volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 15 01:53:05.889: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename volume STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-storage] Volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/volumes.go:42 [It] should be mountable /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/volumes.go:48 STEP: starting configmap-client STEP: Deleting pod configmap-client in namespace volume-7301 May 15 01:58:05.942: INFO: Waiting for pod configmap-client to disappear May 15 01:58:05.944: INFO: Pod configmap-client still exists May 15 01:58:07.945: INFO: Waiting for pod configmap-client to disappear May 15 01:58:07.948: INFO: Pod configmap-client still exists May 15 01:58:09.946: INFO: Waiting for pod configmap-client to disappear May 15 01:58:09.948: INFO: Pod configmap-client no longer exists May 15 01:58:09.948: FAIL: Failed to create client pod: timed out waiting for the condition Full Stack Trace k8s.io/kubernetes/test/e2e/framework/volume.testVolumeClient(0xc002eba6e0, 0xc000f57810, 0xb, 0x4bf3060, 0x9, 0x0, 0x0, 0x0, 0x0, 0x0, ...) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/volume/fixtures.go:519 +0x193 k8s.io/kubernetes/test/e2e/framework/volume.TestVolumeClient(...) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/volume/fixtures.go:504 k8s.io/kubernetes/test/e2e/storage.glob..func29.2.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/volumes.go:111 +0x6c5 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc003230480) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc003230480) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc003230480, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [sig-storage] Volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "volume-7301". STEP: Found 9 events. May 15 01:58:09.959: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for configmap-client: { } Scheduled: Successfully assigned volume-7301/configmap-client to node2 May 15 01:58:09.959: INFO: At 2021-05-15 01:53:07 +0000 UTC - event for configmap-client: {multus } AddedInterface: Add eth0 [10.244.4.241/24] May 15 01:58:09.959: INFO: At 2021-05-15 01:53:07 +0000 UTC - event for configmap-client: {kubelet node2} Pulling: Pulling image "docker.io/library/busybox:1.29" May 15 01:58:09.959: INFO: At 2021-05-15 01:53:08 +0000 UTC - event for configmap-client: {kubelet node2} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 15 01:58:09.959: INFO: At 2021-05-15 01:53:08 +0000 UTC - event for configmap-client: {kubelet node2} Failed: Error: ErrImagePull May 15 01:58:09.959: INFO: At 2021-05-15 01:53:08 +0000 UTC - event for configmap-client: {kubelet node2} SandboxChanged: Pod sandbox changed, it will be killed and re-created. May 15 01:58:09.959: INFO: At 2021-05-15 01:53:10 +0000 UTC - event for configmap-client: {multus } AddedInterface: Add eth0 [10.244.4.242/24] May 15 01:58:09.959: INFO: At 2021-05-15 01:53:10 +0000 UTC - event for configmap-client: {kubelet node2} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 15 01:58:09.959: INFO: At 2021-05-15 01:53:10 +0000 UTC - event for configmap-client: {kubelet node2} Failed: Error: ImagePullBackOff May 15 01:58:09.961: INFO: POD NODE PHASE GRACE CONDITIONS May 15 01:58:09.961: INFO: May 15 01:58:09.966: INFO: Logging node info for node master1 May 15 01:58:09.968: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 163320 0 2021-05-14 19:56:35 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:58:00 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:58:00 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:58:00 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:58:00 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[@ :],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:58:09.969: INFO: Logging kubelet events for node master1 May 15 01:58:09.974: INFO: Logging pods the kubelet thinks is on node master1 May 15 01:58:09.992: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:58:09.992: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:58:09.992: INFO: kubernetes-metrics-scraper-678c97765c-fswrn started at 2021-05-15 00:18:49 +0000 UTC (0+1 container statuses recorded) May 15 01:58:09.992: INFO: Container kubernetes-metrics-scraper ready: true, restart count 0 May 15 01:58:09.992: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:58:09.992: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:58:09.992: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:58:09.992: INFO: Container kube-multus ready: true, restart count 1 May 15 01:58:09.992: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 15 01:58:09.992: INFO: Container docker-registry ready: true, restart count 0 May 15 01:58:09.992: INFO: Container nginx ready: true, restart count 0 May 15 01:58:09.992: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 15 01:58:09.992: INFO: Container nfd-controller ready: true, restart count 0 May 15 01:58:09.992: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:58:09.992: INFO: Container kube-controller-manager ready: true, restart count 2 May 15 01:58:09.992: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:58:09.992: INFO: Init container install-cni ready: true, restart count 0 May 15 01:58:09.992: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:58:09.992: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:58:09.992: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:58:09.992: INFO: Container node-exporter ready: true, restart count 0 May 15 01:58:09.992: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:58:09.992: INFO: Container kube-scheduler ready: true, restart count 0 W0515 01:58:10.005447 32 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:58:10.038: INFO: Latency metrics for node master1 May 15 01:58:10.038: INFO: Logging node info for node master2 May 15 01:58:10.041: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 163355 0 2021-05-14 19:57:04 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234763776 0} {} 196518324Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324599808 0} {} 195629492Ki BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:58:08 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:58:08 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:58:08 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:58:08 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:58:10.042: INFO: Logging kubelet events for node master2 May 15 01:58:10.046: INFO: Logging pods the kubelet thinks is on node master2 May 15 01:58:10.061: INFO: coredns-7677f9bb54-96w24 started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:58:10.061: INFO: Container coredns ready: true, restart count 0 May 15 01:58:10.061: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:58:10.061: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:58:10.061: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:58:10.061: INFO: Container kube-multus ready: true, restart count 1 May 15 01:58:10.061: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:58:10.061: INFO: Container kube-scheduler ready: true, restart count 2 May 15 01:58:10.061: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:58:10.061: INFO: Init container install-cni ready: true, restart count 0 May 15 01:58:10.061: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:58:10.061: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 15 01:58:10.061: INFO: Container autoscaler ready: true, restart count 2 May 15 01:58:10.061: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:58:10.061: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:58:10.061: INFO: Container node-exporter ready: true, restart count 0 May 15 01:58:10.061: INFO: prometheus-operator-5bb8cb9d8f-fqb87 started at 2021-05-15 00:18:49 +0000 UTC (0+2 container statuses recorded) May 15 01:58:10.061: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:58:10.061: INFO: Container prometheus-operator ready: true, restart count 0 May 15 01:58:10.061: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:58:10.061: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:58:10.061: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 15 01:58:10.061: INFO: Container kube-controller-manager ready: true, restart count 2 W0515 01:58:10.074644 32 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:58:10.105: INFO: Latency metrics for node master2 May 15 01:58:10.105: INFO: Logging node info for node master3 May 15 01:58:10.107: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 163344 0 2021-05-14 19:57:14 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{201234767872 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{0 0} {} 0 DecimalSI},memory: {{200324603904 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:58:06 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:58:06 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:58:06 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:58:06 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:58:10.107: INFO: Logging kubelet events for node master3 May 15 01:58:10.111: INFO: Logging pods the kubelet thinks is on node master3 May 15 01:58:10.126: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:58:10.127: INFO: Container kube-apiserver ready: true, restart count 0 May 15 01:58:10.127: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:58:10.127: INFO: Container kube-proxy ready: true, restart count 1 May 15 01:58:10.127: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:58:10.127: INFO: Init container install-cni ready: true, restart count 0 May 15 01:58:10.127: INFO: Container kube-flannel ready: true, restart count 2 May 15 01:58:10.127: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:58:10.127: INFO: Container kube-multus ready: true, restart count 1 May 15 01:58:10.127: INFO: coredns-7677f9bb54-rpj8c started at 2021-05-15 01:12:26 +0000 UTC (0+1 container statuses recorded) May 15 01:58:10.127: INFO: Container coredns ready: true, restart count 0 May 15 01:58:10.127: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:58:10.127: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:58:10.127: INFO: Container node-exporter ready: true, restart count 0 May 15 01:58:10.127: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 15 01:58:10.127: INFO: Container kube-controller-manager ready: true, restart count 3 May 15 01:58:10.127: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 15 01:58:10.127: INFO: Container kube-scheduler ready: true, restart count 3 W0515 01:58:10.139369 32 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:58:10.165: INFO: Latency metrics for node master3 May 15 01:58:10.165: INFO: Logging node info for node node1 May 15 01:58:10.168: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 163353 0 2021-05-14 19:58:21 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1602":"csi-mock-csi-mock-volumes-1602","csi-mock-csi-mock-volumes-253":"csi-mock-csi-mock-volumes-253","csi-mock-csi-mock-volumes-2591":"csi-mock-csi-mock-volumes-2591","csi-mock-csi-mock-volumes-2993":"csi-mock-csi-mock-volumes-2993","csi-mock-csi-mock-volumes-6288":"csi-mock-csi-mock-volumes-6288","csi-mock-csi-mock-volumes-7734":"csi-mock-csi-mock-volumes-7734","csi-mock-csi-mock-volumes-895":"csi-mock-csi-mock-volumes-895","csi-mock-csi-mock-volumes-9232":"csi-mock-csi-mock-volumes-9232","csi-mock-csi-mock-volumes-9474":"csi-mock-csi-mock-volumes-9474","csi-mock-csi-mock-volumes-9728":"csi-mock-csi-mock-volumes-9728","csi-mock-csi-mock-volumes-9809":"csi-mock-csi-mock-volumes-9809"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:00 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:52:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:53:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:58:08 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:58:08 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:58:08 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:58:08 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[@ :],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6c6a0332693a7c456378f6abd2bb40611826c1e1a733cadbdae2daab3125b71c k8s.gcr.io/sig-storage/csi-resizer:v0.5.0],SizeBytes:46041582,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:58:10.169: INFO: Logging kubelet events for node node1 May 15 01:58:10.174: INFO: Logging pods the kubelet thinks is on node node1 May 15 01:58:10.191: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:58:10.191: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:58:10.191: INFO: collectd-mrzps started at 2021-05-15 00:19:22 +0000 UTC (0+3 container statuses recorded) May 15 01:58:10.191: INFO: Container collectd ready: true, restart count 0 May 15 01:58:10.191: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:58:10.191: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:58:10.191: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:58:10.191: INFO: Init container install-cni ready: true, restart count 0 May 15 01:58:10.191: INFO: Container kube-flannel ready: true, restart count 1 May 15 01:58:10.191: INFO: node-feature-discovery-worker-bw8zg started at 2021-05-15 00:18:56 +0000 UTC (0+1 container statuses recorded) May 15 01:58:10.191: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:58:10.191: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:58:10.191: INFO: Container kube-multus ready: true, restart count 1 May 15 01:58:10.191: INFO: prometheus-k8s-0 started at 2021-05-15 00:19:01 +0000 UTC (0+5 container statuses recorded) May 15 01:58:10.191: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 15 01:58:10.191: INFO: Container grafana ready: true, restart count 0 May 15 01:58:10.191: INFO: Container prometheus ready: true, restart count 26 May 15 01:58:10.191: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 15 01:58:10.191: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 15 01:58:10.191: INFO: node-exporter-flvqz started at 2021-05-15 00:18:55 +0000 UTC (0+2 container statuses recorded) May 15 01:58:10.191: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:58:10.191: INFO: Container node-exporter ready: true, restart count 0 May 15 01:58:10.191: INFO: cmk-4s6dm started at 2021-05-15 00:18:54 +0000 UTC (0+2 container statuses recorded) May 15 01:58:10.191: INFO: Container nodereport ready: true, restart count 0 May 15 01:58:10.191: INFO: Container reconcile ready: true, restart count 0 May 15 01:58:10.191: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:58:10.191: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:58:10.191: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-4vxcc started at 2021-05-15 00:19:00 +0000 UTC (0+1 container statuses recorded) May 15 01:58:10.191: INFO: Container kube-sriovdp ready: true, restart count 0 W0515 01:58:10.205217 32 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:58:10.249: INFO: Latency metrics for node node1 May 15 01:58:10.249: INFO: Logging node info for node node2 May 15 01:58:10.252: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 163331 0 2021-05-14 19:58:22 +0000 UTC map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1663":"csi-mock-csi-mock-volumes-1663","csi-mock-csi-mock-volumes-3052":"csi-mock-csi-mock-volumes-3052","csi-mock-csi-mock-volumes-5200":"csi-mock-csi-mock-volumes-5200","csi-mock-csi-mock-volumes-5678":"csi-mock-csi-mock-volumes-5678","csi-mock-csi-mock-volumes-8760":"csi-mock-csi-mock-volumes-8760","csi-mock-csi-mock-volumes-9624":"csi-mock-csi-mock-volumes-9624"} flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {e2e.test Update v1 2021-05-15 01:37:46 +0000 UTC FieldsV1 {"f:status":{"f:capacity":{"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{}}}}} {kube-controller-manager Update v1 2021-05-15 01:52:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubelet Update v1 2021-05-15 01:52:21 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:io.kubernetes.storage.mock/node":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:example.com/fakePTSRes":{},"f:example.com/fakecpu":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{80 0} {} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {} 439913340Ki BinarySI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{201269633024 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {} 3 DecimalSI},cpu: {{77 0} {} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {} 405424133473 DecimalSI},example.com/fakePTSRes: {{10 0} {} 10 DecimalSI},example.com/fakecpu: {{1 3} {} 1k DecimalSI},hugepages-1Gi: {{0 0} {} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {} 4 DecimalSI},memory: {{178884632576 0} {} BinarySI},pods: {{110 0} {} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-15 01:58:03 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-15 01:58:03 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-15 01:58:03 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-15 01:58:03 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[gluster/glusterdynamic-provisioner@sha256:90067cb05a7d217651e84576935934fd9dcff8e6bcdcbaa416bbf36fcd09dbd1 gluster/glusterdynamic-provisioner:v1.0],SizeBytes:373281573,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:78e3393f5fd5ff6c1e5dada2478cfa456fb7164929e573cf9a87bf6532730679 k8s.gcr.io/sig-storage/csi-provisioner:v1.6.0],SizeBytes:48281550,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:2ffa647e8107cfd39e5f464e738dce014c9f5e51b108da36c3ab621048d0bbab k8s.gcr.io/sig-storage/csi-attacher:v2.2.0],SizeBytes:46131354,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:0b4273abac4c241fa3d70aaf52b0d79a133d2737081f4a5c5dea4949f6c45dc3 k8s.gcr.io/sig-storage/mock-driver:v3.1.0],SizeBytes:18687618,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:9622c6a6dac7499a055a382930f4de82905a3c5735c0753f7094115c9c871309 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v1.3.0],SizeBytes:16322467,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 15 01:58:10.253: INFO: Logging kubelet events for node node2 May 15 01:58:10.257: INFO: Logging pods the kubelet thinks is on node node2 May 15 01:58:10.272: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 15 01:58:10.272: INFO: Container nfd-worker ready: true, restart count 0 May 15 01:58:10.272: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 15 01:58:10.272: INFO: Container nodereport ready: true, restart count 0 May 15 01:58:10.272: INFO: Container reconcile ready: true, restart count 0 May 15 01:58:10.272: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 15 01:58:10.272: INFO: Container cmk-webhook ready: true, restart count 0 May 15 01:58:10.272: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 15 01:58:10.272: INFO: Container discover ready: false, restart count 0 May 15 01:58:10.273: INFO: Container init ready: false, restart count 0 May 15 01:58:10.273: INFO: Container install ready: false, restart count 0 May 15 01:58:10.273: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 15 01:58:10.273: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 15 01:58:10.273: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 15 01:58:10.273: INFO: Container kube-proxy ready: true, restart count 2 May 15 01:58:10.273: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 15 01:58:10.273: INFO: Container kube-multus ready: true, restart count 1 May 15 01:58:10.273: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 15 01:58:10.273: INFO: Container nginx-proxy ready: true, restart count 2 May 15 01:58:10.273: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 15 01:58:10.273: INFO: Init container install-cni ready: true, restart count 1 May 15 01:58:10.273: INFO: Container kube-flannel ready: true, restart count 4 May 15 01:58:10.273: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 15 01:58:10.273: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 15 01:58:10.273: INFO: Container node-exporter ready: true, restart count 0 May 15 01:58:10.273: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 15 01:58:10.273: INFO: Container tas-controller ready: true, restart count 0 May 15 01:58:10.273: INFO: Container tas-extender ready: true, restart count 0 May 15 01:58:10.273: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 15 01:58:10.273: INFO: Container collectd ready: true, restart count 0 May 15 01:58:10.273: INFO: Container collectd-exporter ready: true, restart count 0 May 15 01:58:10.273: INFO: Container rbac-proxy ready: true, restart count 0 May 15 01:58:10.273: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 15 01:58:10.273: INFO: Container kube-sriovdp ready: true, restart count 0 W0515 01:58:10.285334 32 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 15 01:58:10.322: INFO: Latency metrics for node node2 May 15 01:58:10.322: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "volume-7301" for this suite. • Failure [304.441 seconds] [sig-storage] Volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23 ConfigMap /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/volumes.go:47 should be mountable [It] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/volumes.go:48 May 15 01:58:09.948: Failed to create client pod: timed out waiting for the condition /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/volume/fixtures.go:519 ------------------------------ {"msg":"FAILED [sig-storage] Volumes ConfigMap should be mountable","total":-1,"completed":4,"skipped":178,"failed":4,"failures":["[sig-storage] PersistentVolumes-local [Volume type: dir-link] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir-link] One pod requesting one prebound PVC should be able to mount volume and write from pod1","[sig-storage] PersistentVolumes-local [Volume type: dir] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2","[sig-storage] Volumes ConfigMap should be mountable"]} May 15 01:58:10.335: INFO: Running AfterSuite actions on all nodes {"msg":"FAILED [sig-storage] PersistentVolumes-local [Volume type: dir-link-bindmounted] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2","total":-1,"completed":3,"skipped":370,"failed":4,"failures":["[sig-storage] PersistentVolumes-local [Volume type: blockfswithoutformat] One pod requesting one prebound PVC should be able to mount volume and read from pod1","[sig-storage] PersistentVolumes-local [Volume type: blockfswithformat] One pod requesting one prebound PVC should be able to mount volume and read from pod1","[sig-storage] PersistentVolumes-local [Volume type: tmpfs] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2","[sig-storage] PersistentVolumes-local [Volume type: dir-link-bindmounted] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2"]} May 15 01:54:14.249: INFO: Running AfterSuite actions on all nodes May 15 01:58:10.377: INFO: Running AfterSuite actions on node 1 May 15 01:58:10.377: INFO: Skipping dumping logs from cluster Summarizing 33 Failures: [Fail] [sig-storage] PersistentVolumes-local [Volume type: blockfswithoutformat] Two pods mounting a local volume one after the other [It] should be able to write from pod1 and read from pod2 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:780 [Fail] [sig-storage] PersistentVolumes-local [Volume type: tmpfs] Two pods mounting a local volume one after the other [It] should be able to write from pod1 and read from pod2 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:780 [Fail] [sig-storage] PersistentVolumes-local [Volume type: dir-link] Two pods mounting a local volume at the same time [It] should be able to write from pod1 and read from pod2 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:727 [Fail] [sig-storage] PersistentVolumes-local [Volume type: dir] One pod requesting one prebound PVC [BeforeEach] should be able to mount volume and write from pod1 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 [Fail] [sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] Two pods mounting a local volume at the same time [It] should be able to write from pod1 and read from pod2 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:740 [Fail] [sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] One pod requesting one prebound PVC [BeforeEach] should be able to mount volume and read from pod1 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 [Fail] [sig-storage] PersistentVolumes-local [Volume type: blockfswithoutformat] One pod requesting one prebound PVC [BeforeEach] should be able to mount volume and read from pod1 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 [Fail] [sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] One pod requesting one prebound PVC [BeforeEach] should be able to mount volume and write from pod1 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 [Fail] [sig-storage] PersistentVolumes-local [Volume type: dir-link-bindmounted] One pod requesting one prebound PVC [BeforeEach] should be able to mount volume and read from pod1 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 [Fail] [sig-storage] PersistentVolumes-local [Volume type: dir-link] One pod requesting one prebound PVC [BeforeEach] should be able to mount volume and write from pod1 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 [Fail] [sig-storage] PersistentVolumes-local [Volume type: blockfswithoutformat] One pod requesting one prebound PVC [BeforeEach] should be able to mount volume and write from pod1 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 [Fail] [sig-storage] PersistentVolumes-local [Volume type: tmpfs] One pod requesting one prebound PVC [BeforeEach] should be able to mount volume and read from pod1 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 [Fail] [sig-storage] PersistentVolumes-local [Volume type: blockfswithformat] One pod requesting one prebound PVC [BeforeEach] should be able to mount volume and read from pod1 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 [Fail] [sig-storage] PersistentVolumes-local [Volume type: dir] Two pods mounting a local volume one after the other [It] should be able to write from pod1 and read from pod2 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:764 [Fail] [sig-storage] PersistentVolumes-local [Volume type: dir-link] Two pods mounting a local volume one after the other [It] should be able to write from pod1 and read from pod2 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:764 [Fail] [sig-storage] PersistentVolumes-local [Volume type: blockfswithformat] Two pods mounting a local volume one after the other [It] should be able to write from pod1 and read from pod2 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:764 [Fail] [sig-storage] PersistentVolumes-local [Volume type: block] One pod requesting one prebound PVC [BeforeEach] should be able to mount volume and write from pod1 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 [Fail] [sig-storage] PersistentVolumes-local [Volume type: blockfswithoutformat] Two pods mounting a local volume at the same time [It] should be able to write from pod1 and read from pod2 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:727 [Fail] [sig-storage] PersistentVolumes-local [Volume type: block] Two pods mounting a local volume one after the other [It] should be able to write from pod1 and read from pod2 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:764 [Fail] [sig-storage] PersistentVolumes-local [Volume type: dir-link-bindmounted] Two pods mounting a local volume at the same time [It] should be able to write from pod1 and read from pod2 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:727 [Fail] [sig-storage] PersistentVolumes-local [Volume type: tmpfs] Two pods mounting a local volume at the same time [It] should be able to write from pod1 and read from pod2 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:727 [Fail] [sig-storage] PersistentVolumes-local [Volume type: blockfswithformat] One pod requesting one prebound PVC [BeforeEach] should be able to mount volume and write from pod1 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 [Fail] [sig-storage] PersistentVolumes-local [Volume type: dir] Two pods mounting a local volume at the same time [It] should be able to write from pod1 and read from pod2 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:727 [Fail] [sig-storage] PersistentVolumes-local [Volume type: dir-bindmounted] Two pods mounting a local volume one after the other [It] should be able to write from pod1 and read from pod2 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:764 [Fail] [sig-storage] PersistentVolumes-local [Volume type: dir] One pod requesting one prebound PVC [BeforeEach] should be able to mount volume and read from pod1 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 [Fail] [sig-storage] PersistentVolumes-local [Volume type: dir-link] One pod requesting one prebound PVC [BeforeEach] should be able to mount volume and read from pod1 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 [Fail] [sig-storage] PersistentVolumes-local [Volume type: dir-link-bindmounted] One pod requesting one prebound PVC [BeforeEach] should be able to mount volume and write from pod1 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 [Fail] [sig-storage] PersistentVolumes-local [Volume type: block] One pod requesting one prebound PVC [BeforeEach] should be able to mount volume and read from pod1 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:214 [Fail] [sig-storage] PersistentVolumes-local [Volume type: dir-link-bindmounted] Two pods mounting a local volume one after the other [It] should be able to write from pod1 and read from pod2 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:764 [Fail] [sig-storage] PersistentVolumes-local [Volume type: blockfswithformat] Two pods mounting a local volume at the same time [It] should be able to write from pod1 and read from pod2 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:727 [Fail] [sig-storage] Subpath Container restart [It] should verify that container can restart successfully after configmaps modified /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:807 [Fail] [sig-storage] PersistentVolumes-local [Volume type: block] Two pods mounting a local volume at the same time [It] should be able to write from pod1 and read from pod2 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/persistent_volumes-local.go:727 [Fail] [sig-storage] Volumes ConfigMap [It] should be mountable /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/volume/fixtures.go:519 Ran 84 of 5484 Specs in 852.323 seconds FAIL! -- 51 Passed | 33 Failed | 0 Pending | 5400 Skipped Ginkgo ran 1 suite in 14m13.741588811s Test Suite Failed