/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 May 14 21:21:41.959: Unexpected error: <*errors.errorString | 0xc004b1edd0>: { s: "expected pod \"downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61\" success: Gave up after waiting 5m0s for pod \"downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61\" to be \"Succeeded or Failed\"", } expected pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61" success: Gave up after waiting 5m0s for pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61" to be "Succeeded or Failed" occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:725 [BeforeEach] [sig-node] Downward API /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 14 21:16:37.382: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename downward-api STEP: Waiting for a default service account to be provisioned in namespace [It] should provide pod name, namespace and IP address as env vars [NodeConformance] [Conformance] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 STEP: Creating a pod to test downward api env vars May 14 21:16:37.412: INFO: Waiting up to 5m0s for pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61" in namespace "downward-api-9440" to be "Succeeded or Failed" May 14 21:16:37.414: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 2.604756ms May 14 21:16:39.418: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 2.00608817s May 14 21:16:41.421: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 4.009129562s May 14 21:16:43.425: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 6.012858339s May 14 21:16:45.428: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 8.01572987s May 14 21:16:47.432: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 10.019683024s May 14 21:16:49.434: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 12.02256814s May 14 21:16:51.437: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 14.025668709s May 14 21:16:53.441: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 16.029061653s May 14 21:16:55.445: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 18.033043542s May 14 21:16:57.447: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 20.035610292s May 14 21:16:59.451: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 22.039000561s May 14 21:17:01.453: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 24.041462548s May 14 21:17:03.457: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 26.044952898s May 14 21:17:05.461: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 28.049267862s May 14 21:17:07.465: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 30.053520887s May 14 21:17:09.468: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 32.056593012s May 14 21:17:11.475: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 34.062704816s May 14 21:17:13.477: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 36.065066719s May 14 21:17:15.480: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 38.068580633s May 14 21:17:17.484: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 40.071924195s May 14 21:17:19.486: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 42.074590015s May 14 21:17:21.489: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 44.077512336s May 14 21:17:23.493: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 46.080807562s May 14 21:17:25.496: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 48.084360165s May 14 21:17:27.500: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 50.087855668s May 14 21:17:29.504: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 52.092081042s May 14 21:17:31.507: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 54.094821295s May 14 21:17:33.510: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 56.098108901s May 14 21:17:35.513: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 58.101006179s May 14 21:17:37.516: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.10459689s May 14 21:17:39.520: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.108515263s May 14 21:17:41.524: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.111763991s May 14 21:17:43.526: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.114615459s May 14 21:17:45.529: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.117585191s May 14 21:17:47.533: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.121079837s May 14 21:17:49.537: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.124794041s May 14 21:17:51.539: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.127513834s May 14 21:17:53.543: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.130838955s May 14 21:17:55.546: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.13411079s May 14 21:17:57.549: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.137117512s May 14 21:17:59.551: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.139487s May 14 21:18:01.554: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.141964704s May 14 21:18:03.557: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.144828637s May 14 21:18:05.559: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.147376232s May 14 21:18:07.562: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.150023998s May 14 21:18:09.566: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.154626372s May 14 21:18:11.569: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.157494066s May 14 21:18:13.572: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.159979905s May 14 21:18:15.575: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.162741295s May 14 21:18:17.579: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.166800125s May 14 21:18:19.582: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.169944663s May 14 21:18:21.585: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.173382013s May 14 21:18:23.589: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.1769445s May 14 21:18:25.593: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.181246813s May 14 21:18:27.597: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.185149369s May 14 21:18:29.600: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.187722378s May 14 21:18:31.603: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.190783807s May 14 21:18:33.605: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.193563742s May 14 21:18:35.609: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.196999285s May 14 21:18:37.611: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.199557006s May 14 21:18:39.615: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.202790276s May 14 21:18:41.617: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.205337621s May 14 21:18:43.621: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.209577344s May 14 21:18:45.625: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.212769055s May 14 21:18:47.629: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.217093006s May 14 21:18:49.633: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.220792542s May 14 21:18:51.636: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.223727375s May 14 21:18:53.639: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.227256325s May 14 21:18:55.643: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.230929157s May 14 21:18:57.646: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.233970045s May 14 21:18:59.649: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.237511909s May 14 21:19:01.653: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.241607286s May 14 21:19:03.659: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.247063036s May 14 21:19:05.664: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.252420509s May 14 21:19:07.669: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.256752619s May 14 21:19:09.673: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.26083945s May 14 21:19:11.676: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.264002957s May 14 21:19:13.679: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.267468591s May 14 21:19:15.684: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.272332118s May 14 21:19:17.687: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.275540877s May 14 21:19:19.692: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.279808377s May 14 21:19:21.696: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.283759696s May 14 21:19:23.699: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.287388208s May 14 21:19:25.705: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.29267771s May 14 21:19:27.711: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.299293672s May 14 21:19:29.715: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.303482251s May 14 21:19:31.719: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.307049203s May 14 21:19:33.722: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.31014577s May 14 21:19:35.726: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.314433822s May 14 21:19:37.730: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.317801227s May 14 21:19:39.733: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.32129455s May 14 21:19:41.737: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.325009377s May 14 21:19:43.741: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.328960448s May 14 21:19:45.745: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.332795679s May 14 21:19:47.748: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.336529108s May 14 21:19:49.751: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.339235862s May 14 21:19:51.754: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.342398755s May 14 21:19:53.757: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.345367793s May 14 21:19:55.761: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.34895468s May 14 21:19:57.764: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.352595156s May 14 21:19:59.768: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.355899167s May 14 21:20:01.774: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.361991126s May 14 21:20:03.779: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.366783384s May 14 21:20:05.782: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.370040873s May 14 21:20:07.785: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.372763038s May 14 21:20:09.788: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.376548512s May 14 21:20:11.797: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.38475281s May 14 21:20:13.800: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.388456611s May 14 21:20:15.806: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.393939765s May 14 21:20:17.809: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.396773942s May 14 21:20:19.813: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.40161456s May 14 21:20:21.817: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.405417737s May 14 21:20:23.821: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.409140933s May 14 21:20:25.826: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.413963465s May 14 21:20:27.829: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.417527037s May 14 21:20:29.832: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.420300249s May 14 21:20:31.836: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.423968174s May 14 21:20:33.839: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.426800444s May 14 21:20:35.842: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.429757287s May 14 21:20:37.845: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.432782096s May 14 21:20:39.848: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.435887415s May 14 21:20:41.851: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.439470393s May 14 21:20:43.855: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.443106857s May 14 21:20:45.858: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.445727863s May 14 21:20:47.861: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.44894774s May 14 21:20:49.863: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.451464536s May 14 21:20:51.867: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.45506597s May 14 21:20:53.869: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.457396215s May 14 21:20:55.872: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.460605025s May 14 21:20:57.876: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.464465467s May 14 21:20:59.879: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.466900566s May 14 21:21:01.882: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.470181317s May 14 21:21:03.885: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.473629352s May 14 21:21:05.890: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.47855353s May 14 21:21:07.893: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.481637945s May 14 21:21:09.897: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.485222748s May 14 21:21:11.900: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.488554848s May 14 21:21:13.903: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.491168764s May 14 21:21:15.906: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.49460645s May 14 21:21:17.909: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.497355472s May 14 21:21:19.913: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.501182319s May 14 21:21:21.917: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.50505439s May 14 21:21:23.921: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.509558506s May 14 21:21:25.925: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.513637618s May 14 21:21:27.928: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.516523779s May 14 21:21:29.931: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.519510115s May 14 21:21:31.936: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.523703322s May 14 21:21:33.939: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.527446929s May 14 21:21:35.942: INFO: Pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.530026517s May 14 21:21:37.948: INFO: Failed to get logs from node "node2" pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61" container "dapi-container": the server rejected our request for an unknown reason (get pods downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61) STEP: delete the pod May 14 21:21:37.954: INFO: Waiting for pod downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61 to disappear May 14 21:21:37.956: INFO: Pod downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61 still exists May 14 21:21:39.956: INFO: Waiting for pod downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61 to disappear May 14 21:21:39.959: INFO: Pod downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61 still exists May 14 21:21:41.957: INFO: Waiting for pod downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61 to disappear May 14 21:21:41.959: INFO: Pod downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61 no longer exists May 14 21:21:41.959: FAIL: Unexpected error: <*errors.errorString | 0xc004b1edd0>: { s: "expected pod \"downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61\" success: Gave up after waiting 5m0s for pod \"downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61\" to be \"Succeeded or Failed\"", } expected pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61" success: Gave up after waiting 5m0s for pod "downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61" to be "Succeeded or Failed" occurred Full Stack Trace k8s.io/kubernetes/test/e2e/framework.(*Framework).testContainerOutputMatcher(0xc000466dc0, 0x4c29f00, 0x15, 0xc0031f1000, 0x0, 0xc00113d1a8, 0x3, 0x3, 0x4de7490) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:725 +0x1ee k8s.io/kubernetes/test/e2e/framework.(*Framework).TestContainerOutputRegexp(...) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:532 k8s.io/kubernetes/test/e2e/common.testDownwardAPIUsingPod(...) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/downward_api.go:425 k8s.io/kubernetes/test/e2e/common.testDownwardAPI(0xc000466dc0, 0xc00128f900, 0x31, 0xc001e79d00, 0x3, 0x3, 0xc00113d1a8, 0x3, 0x3) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/downward_api.go:391 +0x75c k8s.io/kubernetes/test/e2e/common.glob..func5.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/downward_api.go:80 +0x4e8 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc00341a900) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc00341a900) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc00341a900, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [sig-node] Downward API /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "downward-api-9440". STEP: Found 10 events. May 14 21:21:41.965: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61: { } Scheduled: Successfully assigned downward-api-9440/downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61 to node2 May 14 21:21:41.965: INFO: At 2021-05-14 21:16:39 +0000 UTC - event for downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61: {multus } AddedInterface: Add eth0 [10.244.4.71/24] May 14 21:21:41.965: INFO: At 2021-05-14 21:16:39 +0000 UTC - event for downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61: {kubelet node2} Pulling: Pulling image "docker.io/library/busybox:1.29" May 14 21:21:41.965: INFO: At 2021-05-14 21:16:39 +0000 UTC - event for downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61: {kubelet node2} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 14 21:21:41.965: INFO: At 2021-05-14 21:16:39 +0000 UTC - event for downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61: {kubelet node2} Failed: Error: ErrImagePull May 14 21:21:41.966: INFO: At 2021-05-14 21:16:40 +0000 UTC - event for downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61: {kubelet node2} SandboxChanged: Pod sandbox changed, it will be killed and re-created. May 14 21:21:41.966: INFO: At 2021-05-14 21:16:43 +0000 UTC - event for downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61: {multus } AddedInterface: Add eth0 [10.244.4.72/24] May 14 21:21:41.966: INFO: At 2021-05-14 21:16:43 +0000 UTC - event for downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61: {kubelet node2} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 14 21:21:41.966: INFO: At 2021-05-14 21:16:43 +0000 UTC - event for downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61: {kubelet node2} Failed: Error: ImagePullBackOff May 14 21:21:41.966: INFO: At 2021-05-14 21:16:45 +0000 UTC - event for downward-api-44c5b413-a1ee-4173-a91e-6ba172df4e61: {multus } AddedInterface: Add eth0 [10.244.4.73/24] May 14 21:21:41.968: INFO: POD NODE PHASE GRACE CONDITIONS May 14 21:21:41.968: INFO: May 14 21:21:41.972: INFO: Logging node info for node master1 May 14 21:21:41.975: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 26514 0 2021-05-14 19:56:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-14 21:21:35 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-14 21:21:35 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-14 21:21:35 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-14 21:21:35 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 14 21:21:41.975: INFO: Logging kubelet events for node master1 May 14 21:21:41.978: INFO: Logging pods the kubelet thinks is on node master1 May 14 21:21:41.999: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 14 21:21:41.999: INFO: Container kube-controller-manager ready: true, restart count 2 May 14 21:21:41.999: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 14 21:21:41.999: INFO: Init container install-cni ready: true, restart count 0 May 14 21:21:41.999: INFO: Container kube-flannel ready: true, restart count 1 May 14 21:21:41.999: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 14 21:21:41.999: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 14 21:21:41.999: INFO: Container node-exporter ready: true, restart count 0 May 14 21:21:41.999: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 14 21:21:41.999: INFO: Container kube-scheduler ready: true, restart count 0 May 14 21:21:41.999: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 14 21:21:41.999: INFO: Container docker-registry ready: true, restart count 0 May 14 21:21:41.999: INFO: Container nginx ready: true, restart count 0 May 14 21:21:41.999: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 14 21:21:41.999: INFO: Container nfd-controller ready: true, restart count 0 May 14 21:21:41.999: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 14 21:21:41.999: INFO: Container kube-apiserver ready: true, restart count 0 May 14 21:21:41.999: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 14 21:21:41.999: INFO: Container kube-proxy ready: true, restart count 2 May 14 21:21:41.999: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 14 21:21:41.999: INFO: Container kube-multus ready: true, restart count 1 May 14 21:21:41.999: INFO: coredns-7677f9bb54-mlcmv started at 2021-05-14 19:59:27 +0000 UTC (0+1 container statuses recorded) May 14 21:21:41.999: INFO: Container coredns ready: true, restart count 1 W0514 21:21:42.011366 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 14 21:21:42.039: INFO: Latency metrics for node master1 May 14 21:21:42.039: INFO: Logging node info for node master2 May 14 21:21:42.042: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 26515 0 2021-05-14 19:57:04 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-14 21:21:35 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-14 21:21:35 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-14 21:21:35 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-14 21:21:35 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 14 21:21:42.043: INFO: Logging kubelet events for node master2 May 14 21:21:42.044: INFO: Logging pods the kubelet thinks is on node master2 May 14 21:21:42.053: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.053: INFO: Container kube-proxy ready: true, restart count 2 May 14 21:21:42.053: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 14 21:21:42.053: INFO: Init container install-cni ready: true, restart count 0 May 14 21:21:42.053: INFO: Container kube-flannel ready: true, restart count 1 May 14 21:21:42.053: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.053: INFO: Container kube-multus ready: true, restart count 1 May 14 21:21:42.053: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.053: INFO: Container autoscaler ready: true, restart count 2 May 14 21:21:42.053: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 14 21:21:42.053: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 14 21:21:42.053: INFO: Container node-exporter ready: true, restart count 0 May 14 21:21:42.053: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.053: INFO: Container kube-apiserver ready: true, restart count 0 May 14 21:21:42.053: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.053: INFO: Container kube-controller-manager ready: true, restart count 2 May 14 21:21:42.053: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.053: INFO: Container kube-scheduler ready: true, restart count 2 W0514 21:21:42.064393 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 14 21:21:42.093: INFO: Latency metrics for node master2 May 14 21:21:42.093: INFO: Logging node info for node master3 May 14 21:21:42.096: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 26419 0 2021-05-14 19:57:14 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-14 21:21:32 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-14 21:21:32 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-14 21:21:32 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-14 21:21:32 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 14 21:21:42.096: INFO: Logging kubelet events for node master3 May 14 21:21:42.098: INFO: Logging pods the kubelet thinks is on node master3 May 14 21:21:42.106: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 14 21:21:42.106: INFO: Init container install-cni ready: true, restart count 0 May 14 21:21:42.106: INFO: Container kube-flannel ready: true, restart count 2 May 14 21:21:42.106: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.106: INFO: Container kube-multus ready: true, restart count 1 May 14 21:21:42.106: INFO: coredns-7677f9bb54-qhzrv started at 2021-05-14 19:59:32 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.106: INFO: Container coredns ready: true, restart count 2 May 14 21:21:42.106: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 14 21:21:42.106: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 14 21:21:42.106: INFO: Container node-exporter ready: true, restart count 0 May 14 21:21:42.106: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.106: INFO: Container kube-controller-manager ready: true, restart count 3 May 14 21:21:42.106: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.106: INFO: Container kube-scheduler ready: true, restart count 3 May 14 21:21:42.106: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.106: INFO: Container kube-apiserver ready: true, restart count 0 May 14 21:21:42.106: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.106: INFO: Container kube-proxy ready: true, restart count 1 W0514 21:21:42.119120 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 14 21:21:42.147: INFO: Latency metrics for node master3 May 14 21:21:42.147: INFO: Logging node info for node node1 May 14 21:21:42.150: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 26431 0 2021-05-14 19:58:21 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-05-14 19:58:21 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-05-14 20:08:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269633024 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884632576 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-14 21:21:32 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-14 21:21:32 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-14 21:21:32 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-14 21:21:32 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 14 21:21:42.151: INFO: Logging kubelet events for node node1 May 14 21:21:42.153: INFO: Logging pods the kubelet thinks is on node node1 May 14 21:21:42.168: INFO: prometheus-k8s-0 started at 2021-05-14 20:09:58 +0000 UTC (0+5 container statuses recorded) May 14 21:21:42.168: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 14 21:21:42.168: INFO: Container grafana ready: true, restart count 0 May 14 21:21:42.168: INFO: Container prometheus ready: true, restart count 1 May 14 21:21:42.168: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 14 21:21:42.168: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 14 21:21:42.168: INFO: kubernetes-metrics-scraper-678c97765c-pqnd6 started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.168: INFO: Container kubernetes-metrics-scraper ready: true, restart count 1 May 14 21:21:42.168: INFO: node-feature-discovery-worker-nvg9t started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.168: INFO: Container nfd-worker ready: true, restart count 0 May 14 21:21:42.168: INFO: cmk-9nqjc started at 2021-05-14 20:09:03 +0000 UTC (0+2 container statuses recorded) May 14 21:21:42.168: INFO: Container nodereport ready: true, restart count 0 May 14 21:21:42.168: INFO: Container reconcile ready: true, restart count 0 May 14 21:21:42.168: INFO: node-exporter-r62rn started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 14 21:21:42.168: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 14 21:21:42.168: INFO: Container node-exporter ready: true, restart count 0 May 14 21:21:42.168: INFO: server started at 2021-05-14 21:19:02 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.168: INFO: Container agnhost-container ready: true, restart count 0 May 14 21:21:42.168: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 14 21:21:42.168: INFO: Init container install-cni ready: true, restart count 0 May 14 21:21:42.168: INFO: Container kube-flannel ready: true, restart count 1 May 14 21:21:42.168: INFO: prometheus-operator-5bb8cb9d8f-4lq8j started at 2021-05-14 20:09:49 +0000 UTC (0+2 container statuses recorded) May 14 21:21:42.168: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 14 21:21:42.168: INFO: Container prometheus-operator ready: true, restart count 0 May 14 21:21:42.168: INFO: pod-service-account-000efc4a-5ea3-4487-8e6e-e91e49600d7f started at 2021-05-14 21:21:40 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.168: INFO: Container test ready: false, restart count 0 May 14 21:21:42.168: INFO: affinity-nodeport-timeout-7lrrs started at 2021-05-14 21:21:30 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.168: INFO: Container affinity-nodeport-timeout ready: true, restart count 0 May 14 21:21:42.168: INFO: terminate-cmd-rpa1a2dad09-c3ba-4d68-abad-6b6d5e423721 started at 2021-05-14 21:21:35 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.168: INFO: Container terminate-cmd-rpa ready: false, restart count 0 May 14 21:21:42.168: INFO: busybox-scheduling-45397666-5b4e-4380-bf13-3d2866cc9b0c started at 2021-05-14 21:21:01 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.168: INFO: Container busybox-scheduling-45397666-5b4e-4380-bf13-3d2866cc9b0c ready: false, restart count 0 May 14 21:21:42.168: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.168: INFO: Container kube-proxy ready: true, restart count 2 May 14 21:21:42.168: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.168: INFO: Container kube-multus ready: true, restart count 1 May 14 21:21:42.168: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-bw262 started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.168: INFO: Container kube-sriovdp ready: true, restart count 0 May 14 21:21:42.168: INFO: collectd-qhp77 started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 14 21:21:42.168: INFO: Container collectd ready: true, restart count 0 May 14 21:21:42.168: INFO: Container collectd-exporter ready: true, restart count 0 May 14 21:21:42.168: INFO: Container rbac-proxy ready: true, restart count 0 May 14 21:21:42.168: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.168: INFO: Container nginx-proxy ready: true, restart count 2 May 14 21:21:42.168: INFO: cmk-init-discover-node1-g6c6q started at 2021-05-14 20:08:21 +0000 UTC (0+3 container statuses recorded) May 14 21:21:42.168: INFO: Container discover ready: false, restart count 0 May 14 21:21:42.168: INFO: Container init ready: false, restart count 0 May 14 21:21:42.168: INFO: Container install ready: false, restart count 0 May 14 21:21:42.168: INFO: pod-2c1078b1-e23e-47a6-b188-55912f964c05 started at 2021-05-14 21:21:41 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.168: INFO: Container test-container ready: false, restart count 0 W0514 21:21:42.181224 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 14 21:21:42.212: INFO: Latency metrics for node node1 May 14 21:21:42.212: INFO: Logging node info for node node2 May 14 21:21:42.216: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 26439 0 2021-05-14 19:58:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-05-14 20:08:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269633024 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884632576 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-14 21:21:33 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-14 21:21:33 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-14 21:21:33 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-14 21:21:33 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 14 21:21:42.216: INFO: Logging kubelet events for node node2 May 14 21:21:42.218: INFO: Logging pods the kubelet thinks is on node node2 May 14 21:21:42.228: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 14 21:21:42.228: INFO: Init container install-cni ready: true, restart count 1 May 14 21:21:42.228: INFO: Container kube-flannel ready: true, restart count 4 May 14 21:21:42.228: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.228: INFO: Container cmk-webhook ready: true, restart count 0 May 14 21:21:42.228: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 14 21:21:42.228: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 14 21:21:42.228: INFO: Container node-exporter ready: true, restart count 0 May 14 21:21:42.228: INFO: affinity-nodeport-timeout-l5nb5 started at 2021-05-14 21:21:30 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.228: INFO: Container affinity-nodeport-timeout ready: true, restart count 0 May 14 21:21:42.228: INFO: ss2-0 started at 2021-05-14 21:16:52 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.228: INFO: Container webserver ready: false, restart count 0 May 14 21:21:42.228: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 14 21:21:42.228: INFO: Container discover ready: false, restart count 0 May 14 21:21:42.228: INFO: Container init ready: false, restart count 0 May 14 21:21:42.228: INFO: Container install ready: false, restart count 0 May 14 21:21:42.228: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 14 21:21:42.228: INFO: Container tas-controller ready: true, restart count 0 May 14 21:21:42.228: INFO: Container tas-extender ready: true, restart count 0 May 14 21:21:42.228: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 14 21:21:42.228: INFO: Container collectd ready: true, restart count 0 May 14 21:21:42.228: INFO: Container collectd-exporter ready: true, restart count 0 May 14 21:21:42.228: INFO: Container rbac-proxy ready: true, restart count 0 May 14 21:21:42.228: INFO: termination-message-containere6005221-b7c3-46a0-8e25-68b7e59305ac started at 2021-05-14 21:19:05 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.228: INFO: Container termination-message-container ready: false, restart count 0 May 14 21:21:42.228: INFO: tester started at 2021-05-14 21:19:06 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.228: INFO: Container tester ready: false, restart count 0 May 14 21:21:42.228: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.228: INFO: Container kube-proxy ready: true, restart count 2 May 14 21:21:42.228: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.228: INFO: Container kube-multus ready: true, restart count 1 May 14 21:21:42.228: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.228: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 14 21:21:42.228: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.228: INFO: Container kube-sriovdp ready: true, restart count 0 May 14 21:21:42.228: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.228: INFO: Container nginx-proxy ready: true, restart count 2 May 14 21:21:42.228: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.228: INFO: Container nfd-worker ready: true, restart count 0 May 14 21:21:42.228: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 14 21:21:42.228: INFO: Container nodereport ready: true, restart count 0 May 14 21:21:42.228: INFO: Container reconcile ready: true, restart count 0 May 14 21:21:42.228: INFO: var-expansion-be98b9be-755e-4294-b561-fa224913d270 started at 2021-05-14 21:21:18 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.228: INFO: Container dapi-container ready: false, restart count 0 May 14 21:21:42.228: INFO: affinity-nodeport-timeout-5xs6l started at 2021-05-14 21:21:30 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.228: INFO: Container affinity-nodeport-timeout ready: true, restart count 0 May 14 21:21:42.228: INFO: execpod-affinityrj2vk started at 2021-05-14 21:21:36 +0000 UTC (0+1 container statuses recorded) May 14 21:21:42.228: INFO: Container agnhost-container ready: true, restart count 0 W0514 21:21:42.238802 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 14 21:21:42.269: INFO: Latency metrics for node node2 May 14 21:21:42.269: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "downward-api-9440" for this suite. /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 May 14 21:26:42.386: Expected <*errors.errorString | 0xc0002ba200>: { s: "timed out waiting for the condition", } to be nil /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/init_container.go:432 [BeforeEach] [k8s.io] InitContainer [NodeConformance] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 14 21:21:42.350: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename init-container STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [k8s.io] InitContainer [NodeConformance] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/init_container.go:162 [It] should not start app containers if init containers fail on a RestartAlways pod [Conformance] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 STEP: creating the pod May 14 21:21:42.369: INFO: PodSpec: initContainers in spec.initContainers May 14 21:26:42.386: FAIL: Expected <*errors.errorString | 0xc0002ba200>: { s: "timed out waiting for the condition", } to be nil Full Stack Trace k8s.io/kubernetes/test/e2e/common.glob..func11.4() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/init_container.go:432 +0xcaf k8s.io/kubernetes/test/e2e.RunE2ETests(0xc00341a900) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc00341a900) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc00341a900, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [k8s.io] InitContainer [NodeConformance] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "init-container-3734". STEP: Found 7 events. May 14 21:26:42.391: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-init-7ea4fb42-75f4-4d54-a036-96d88aab0965: { } Scheduled: Successfully assigned init-container-3734/pod-init-7ea4fb42-75f4-4d54-a036-96d88aab0965 to node2 May 14 21:26:42.391: INFO: At 2021-05-14 21:21:45 +0000 UTC - event for pod-init-7ea4fb42-75f4-4d54-a036-96d88aab0965: {multus } AddedInterface: Add eth0 [10.244.4.99/24] May 14 21:26:42.391: INFO: At 2021-05-14 21:21:45 +0000 UTC - event for pod-init-7ea4fb42-75f4-4d54-a036-96d88aab0965: {kubelet node2} Pulling: Pulling image "docker.io/library/busybox:1.29" May 14 21:26:42.391: INFO: At 2021-05-14 21:21:46 +0000 UTC - event for pod-init-7ea4fb42-75f4-4d54-a036-96d88aab0965: {kubelet node2} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 14 21:26:42.391: INFO: At 2021-05-14 21:21:46 +0000 UTC - event for pod-init-7ea4fb42-75f4-4d54-a036-96d88aab0965: {kubelet node2} Failed: Error: ErrImagePull May 14 21:26:42.391: INFO: At 2021-05-14 21:21:47 +0000 UTC - event for pod-init-7ea4fb42-75f4-4d54-a036-96d88aab0965: {kubelet node2} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 14 21:26:42.391: INFO: At 2021-05-14 21:21:47 +0000 UTC - event for pod-init-7ea4fb42-75f4-4d54-a036-96d88aab0965: {kubelet node2} Failed: Error: ImagePullBackOff May 14 21:26:42.394: INFO: POD NODE PHASE GRACE CONDITIONS May 14 21:26:42.394: INFO: pod-init-7ea4fb42-75f4-4d54-a036-96d88aab0965 node2 Pending [{Initialized False 0001-01-01 00:00:00 +0000 UTC 2021-05-14 21:21:42 +0000 UTC ContainersNotInitialized containers with incomplete status: [init1 init2]} {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-14 21:21:42 +0000 UTC ContainersNotReady containers with unready status: [run1]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-14 21:21:42 +0000 UTC ContainersNotReady containers with unready status: [run1]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-14 21:21:42 +0000 UTC }] May 14 21:26:42.394: INFO: May 14 21:26:42.398: INFO: Logging node info for node master1 May 14 21:26:42.401: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 28832 0 2021-05-14 19:56:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-14 21:26:36 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-14 21:26:36 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-14 21:26:36 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-14 21:26:36 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 14 21:26:42.401: INFO: Logging kubelet events for node master1 May 14 21:26:42.403: INFO: Logging pods the kubelet thinks is on node master1 May 14 21:26:42.412: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 14 21:26:42.412: INFO: Init container install-cni ready: true, restart count 0 May 14 21:26:42.412: INFO: Container kube-flannel ready: true, restart count 1 May 14 21:26:42.412: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 14 21:26:42.412: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 14 21:26:42.412: INFO: Container node-exporter ready: true, restart count 0 May 14 21:26:42.412: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.412: INFO: Container kube-scheduler ready: true, restart count 0 May 14 21:26:42.412: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.412: INFO: Container kube-controller-manager ready: true, restart count 2 May 14 21:26:42.412: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.412: INFO: Container kube-proxy ready: true, restart count 2 May 14 21:26:42.412: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.412: INFO: Container kube-multus ready: true, restart count 1 May 14 21:26:42.412: INFO: coredns-7677f9bb54-mlcmv started at 2021-05-14 19:59:27 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.412: INFO: Container coredns ready: true, restart count 1 May 14 21:26:42.412: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 14 21:26:42.412: INFO: Container docker-registry ready: true, restart count 0 May 14 21:26:42.412: INFO: Container nginx ready: true, restart count 0 May 14 21:26:42.412: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.412: INFO: Container nfd-controller ready: true, restart count 0 May 14 21:26:42.412: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.412: INFO: Container kube-apiserver ready: true, restart count 0 W0514 21:26:42.424213 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 14 21:26:42.448: INFO: Latency metrics for node master1 May 14 21:26:42.448: INFO: Logging node info for node master2 May 14 21:26:42.451: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 28831 0 2021-05-14 19:57:04 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-14 21:26:36 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-14 21:26:36 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-14 21:26:36 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-14 21:26:36 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 14 21:26:42.451: INFO: Logging kubelet events for node master2 May 14 21:26:42.453: INFO: Logging pods the kubelet thinks is on node master2 May 14 21:26:42.462: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.462: INFO: Container kube-multus ready: true, restart count 1 May 14 21:26:42.462: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.462: INFO: Container autoscaler ready: true, restart count 2 May 14 21:26:42.462: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 14 21:26:42.462: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 14 21:26:42.462: INFO: Container node-exporter ready: true, restart count 0 May 14 21:26:42.462: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.462: INFO: Container kube-apiserver ready: true, restart count 0 May 14 21:26:42.462: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.462: INFO: Container kube-controller-manager ready: true, restart count 2 May 14 21:26:42.462: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.462: INFO: Container kube-scheduler ready: true, restart count 2 May 14 21:26:42.462: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.462: INFO: Container kube-proxy ready: true, restart count 2 May 14 21:26:42.462: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 14 21:26:42.462: INFO: Init container install-cni ready: true, restart count 0 May 14 21:26:42.462: INFO: Container kube-flannel ready: true, restart count 1 W0514 21:26:42.476094 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 14 21:26:42.504: INFO: Latency metrics for node master2 May 14 21:26:42.504: INFO: Logging node info for node master3 May 14 21:26:42.506: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 28797 0 2021-05-14 19:57:14 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-14 21:26:33 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-14 21:26:33 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-14 21:26:33 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-14 21:26:33 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 14 21:26:42.507: INFO: Logging kubelet events for node master3 May 14 21:26:42.508: INFO: Logging pods the kubelet thinks is on node master3 May 14 21:26:42.516: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.516: INFO: Container kube-scheduler ready: true, restart count 3 May 14 21:26:42.516: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.516: INFO: Container kube-apiserver ready: true, restart count 0 May 14 21:26:42.516: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.516: INFO: Container kube-proxy ready: true, restart count 1 May 14 21:26:42.516: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 14 21:26:42.516: INFO: Init container install-cni ready: true, restart count 0 May 14 21:26:42.516: INFO: Container kube-flannel ready: true, restart count 2 May 14 21:26:42.516: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.516: INFO: Container kube-multus ready: true, restart count 1 May 14 21:26:42.516: INFO: coredns-7677f9bb54-qhzrv started at 2021-05-14 19:59:32 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.516: INFO: Container coredns ready: true, restart count 2 May 14 21:26:42.516: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 14 21:26:42.516: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 14 21:26:42.516: INFO: Container node-exporter ready: true, restart count 0 May 14 21:26:42.516: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.516: INFO: Container kube-controller-manager ready: true, restart count 3 W0514 21:26:42.529388 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 14 21:26:42.551: INFO: Latency metrics for node master3 May 14 21:26:42.551: INFO: Logging node info for node node1 May 14 21:26:42.554: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 28802 0 2021-05-14 19:58:21 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-05-14 19:58:21 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-05-14 20:08:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269633024 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884632576 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-14 21:26:34 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-14 21:26:34 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-14 21:26:34 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-14 21:26:34 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 14 21:26:42.555: INFO: Logging kubelet events for node node1 May 14 21:26:42.557: INFO: Logging pods the kubelet thinks is on node node1 May 14 21:26:42.572: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.572: INFO: Container nginx-proxy ready: true, restart count 2 May 14 21:26:42.572: INFO: cmk-init-discover-node1-g6c6q started at 2021-05-14 20:08:21 +0000 UTC (0+3 container statuses recorded) May 14 21:26:42.572: INFO: Container discover ready: false, restart count 0 May 14 21:26:42.572: INFO: Container init ready: false, restart count 0 May 14 21:26:42.572: INFO: Container install ready: false, restart count 0 May 14 21:26:42.572: INFO: node-exporter-r62rn started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 14 21:26:42.572: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 14 21:26:42.572: INFO: Container node-exporter ready: true, restart count 0 May 14 21:26:42.572: INFO: prometheus-k8s-0 started at 2021-05-14 20:09:58 +0000 UTC (0+5 container statuses recorded) May 14 21:26:42.572: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 14 21:26:42.572: INFO: Container grafana ready: true, restart count 0 May 14 21:26:42.572: INFO: Container prometheus ready: true, restart count 1 May 14 21:26:42.572: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 14 21:26:42.572: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 14 21:26:42.572: INFO: fail-once-local-frcf7 started at 2021-05-14 21:26:40 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.572: INFO: Container c ready: false, restart count 0 May 14 21:26:42.572: INFO: kubernetes-metrics-scraper-678c97765c-pqnd6 started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.572: INFO: Container kubernetes-metrics-scraper ready: true, restart count 1 May 14 21:26:42.572: INFO: node-feature-discovery-worker-nvg9t started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.572: INFO: Container nfd-worker ready: true, restart count 0 May 14 21:26:42.572: INFO: cmk-9nqjc started at 2021-05-14 20:09:03 +0000 UTC (0+2 container statuses recorded) May 14 21:26:42.572: INFO: Container nodereport ready: true, restart count 0 May 14 21:26:42.572: INFO: Container reconcile ready: true, restart count 0 May 14 21:26:42.572: INFO: pod-sharedvolume-9d40da78-d9a0-4470-aff6-7399b2ddf7ef started at 2021-05-14 21:24:20 +0000 UTC (0+2 container statuses recorded) May 14 21:26:42.572: INFO: Container busybox-main-container ready: false, restart count 0 May 14 21:26:42.572: INFO: Container busybox-sub-container ready: false, restart count 0 May 14 21:26:42.572: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 14 21:26:42.572: INFO: Init container install-cni ready: true, restart count 0 May 14 21:26:42.572: INFO: Container kube-flannel ready: true, restart count 1 May 14 21:26:42.572: INFO: prometheus-operator-5bb8cb9d8f-4lq8j started at 2021-05-14 20:09:49 +0000 UTC (0+2 container statuses recorded) May 14 21:26:42.572: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 14 21:26:42.572: INFO: Container prometheus-operator ready: true, restart count 0 May 14 21:26:42.572: INFO: pod-service-account-000efc4a-5ea3-4487-8e6e-e91e49600d7f started at 2021-05-14 21:21:40 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.572: INFO: Container test ready: false, restart count 0 May 14 21:26:42.572: INFO: collectd-qhp77 started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 14 21:26:42.572: INFO: Container collectd ready: true, restart count 0 May 14 21:26:42.572: INFO: Container collectd-exporter ready: true, restart count 0 May 14 21:26:42.572: INFO: Container rbac-proxy ready: true, restart count 0 May 14 21:26:42.572: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.572: INFO: Container kube-proxy ready: true, restart count 2 May 14 21:26:42.572: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.572: INFO: Container kube-multus ready: true, restart count 1 May 14 21:26:42.572: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-bw262 started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.572: INFO: Container kube-sriovdp ready: true, restart count 0 W0514 21:26:42.584070 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 14 21:26:42.622: INFO: Latency metrics for node node1 May 14 21:26:42.622: INFO: Logging node info for node node2 May 14 21:26:42.625: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 28803 0 2021-05-14 19:58:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-05-14 20:08:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269633024 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884632576 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-14 21:26:34 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-14 21:26:34 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-14 21:26:34 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-14 21:26:34 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 14 21:26:42.625: INFO: Logging kubelet events for node node2 May 14 21:26:42.627: INFO: Logging pods the kubelet thinks is on node node2 May 14 21:26:42.641: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 14 21:26:42.641: INFO: Container discover ready: false, restart count 0 May 14 21:26:42.641: INFO: Container init ready: false, restart count 0 May 14 21:26:42.641: INFO: Container install ready: false, restart count 0 May 14 21:26:42.641: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 14 21:26:42.641: INFO: Container tas-controller ready: true, restart count 0 May 14 21:26:42.641: INFO: Container tas-extender ready: true, restart count 0 May 14 21:26:42.641: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 14 21:26:42.641: INFO: Container collectd ready: true, restart count 0 May 14 21:26:42.641: INFO: Container collectd-exporter ready: true, restart count 0 May 14 21:26:42.641: INFO: Container rbac-proxy ready: true, restart count 0 May 14 21:26:42.641: INFO: fail-once-local-vsfv6 started at 2021-05-14 21:26:40 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.641: INFO: Container c ready: false, restart count 0 May 14 21:26:42.641: INFO: pod-secrets-637d8d95-1b69-464b-ba71-4883dbbc8178 started at 2021-05-14 21:21:45 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.641: INFO: Container secret-env-test ready: false, restart count 0 May 14 21:26:42.641: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.641: INFO: Container kube-proxy ready: true, restart count 2 May 14 21:26:42.641: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.641: INFO: Container kube-multus ready: true, restart count 1 May 14 21:26:42.641: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.641: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 14 21:26:42.641: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.641: INFO: Container kube-sriovdp ready: true, restart count 0 May 14 21:26:42.641: INFO: liveness-9fa58f7c-ac7d-4238-9615-5e6e86456837 started at 2021-05-14 21:25:39 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.641: INFO: Container liveness ready: true, restart count 0 May 14 21:26:42.641: INFO: downwardapi-volume-7b009d45-62a0-4b12-82c5-70e98dd51713 started at 2021-05-14 21:26:40 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.641: INFO: Container client-container ready: false, restart count 0 May 14 21:26:42.641: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.642: INFO: Container nginx-proxy ready: true, restart count 2 May 14 21:26:42.642: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.642: INFO: Container nfd-worker ready: true, restart count 0 May 14 21:26:42.642: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 14 21:26:42.642: INFO: Container nodereport ready: true, restart count 0 May 14 21:26:42.642: INFO: Container reconcile ready: true, restart count 0 May 14 21:26:42.642: INFO: busybox-user-65534-69c07549-5cd7-4a38-80d2-7e5291ec2832 started at 2021-05-14 21:23:59 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.642: INFO: Container busybox-user-65534-69c07549-5cd7-4a38-80d2-7e5291ec2832 ready: false, restart count 0 May 14 21:26:42.642: INFO: pod-init-7ea4fb42-75f4-4d54-a036-96d88aab0965 started at 2021-05-14 21:21:42 +0000 UTC (2+1 container statuses recorded) May 14 21:26:42.642: INFO: Init container init1 ready: false, restart count 0 May 14 21:26:42.642: INFO: Init container init2 ready: false, restart count 0 May 14 21:26:42.642: INFO: Container run1 ready: false, restart count 0 May 14 21:26:42.642: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 14 21:26:42.642: INFO: Init container install-cni ready: true, restart count 1 May 14 21:26:42.642: INFO: Container kube-flannel ready: true, restart count 4 May 14 21:26:42.642: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.642: INFO: Container cmk-webhook ready: true, restart count 0 May 14 21:26:42.642: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 14 21:26:42.642: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 14 21:26:42.642: INFO: Container node-exporter ready: true, restart count 0 May 14 21:26:42.642: INFO: test-rollover-controller-4ds9r started at 2021-05-14 21:24:35 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.642: INFO: Container httpd ready: false, restart count 0 May 14 21:26:42.642: INFO: ss2-0 started at 2021-05-14 21:16:52 +0000 UTC (0+1 container statuses recorded) May 14 21:26:42.642: INFO: Container webserver ready: false, restart count 0 W0514 21:26:42.653973 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 14 21:26:42.697: INFO: Latency metrics for node node2 May 14 21:26:42.697: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "init-container-3734" for this suite. /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 May 14 21:37:08.448: Failed waiting for pods to enter running: timed out waiting for the condition /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/statefulset/wait.go:58 [BeforeEach] [sig-apps] StatefulSet /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 14 21:27:08.413: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename statefulset STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-apps] StatefulSet /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/apps/statefulset.go:88 [BeforeEach] [k8s.io] Basic StatefulSet functionality [StatefulSetBasic] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/apps/statefulset.go:103 STEP: Creating service test in namespace statefulset-8972 [It] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 STEP: Initializing watcher for selector baz=blah,foo=bar STEP: Creating stateful set ss in namespace statefulset-8972 STEP: Waiting until all stateful set ss replicas will be running in namespace statefulset-8972 May 14 21:27:08.441: INFO: Found 0 stateful pods, waiting for 1 May 14 21:27:18.446: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:27:28.446: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:27:38.444: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:27:48.446: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:27:58.444: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:28:08.444: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:28:18.445: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:28:28.445: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:28:38.444: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:28:48.444: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:28:58.445: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:29:08.446: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:29:18.445: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:29:28.446: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:29:38.443: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:29:48.444: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:29:58.446: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:30:08.446: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:30:18.445: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:30:28.445: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:30:38.444: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:30:48.446: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:30:58.446: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:31:08.445: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:31:18.447: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:31:28.444: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:31:38.444: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:31:48.444: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:31:58.445: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:32:08.445: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:32:18.444: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:32:28.446: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:32:38.445: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:32:48.446: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:32:58.447: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:33:08.446: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:33:18.446: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:33:28.446: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:33:38.447: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:33:48.445: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:33:58.445: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:34:08.446: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:34:18.446: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:34:28.447: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:34:38.445: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:34:48.444: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:34:58.444: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:35:08.446: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:35:18.445: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:35:28.444: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:35:38.445: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:35:48.444: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:35:58.444: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:36:08.444: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:36:18.446: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:36:28.446: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:36:38.444: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:36:48.446: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:36:58.445: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:37:08.445: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:37:08.448: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false May 14 21:37:08.448: FAIL: Failed waiting for pods to enter running: timed out waiting for the condition Full Stack Trace k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning(0x54075e0, 0xc001e034a0, 0x100000001, 0xc001470500) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/statefulset/wait.go:58 +0x10e k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/statefulset/wait.go:80 k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/apps/statefulset.go:622 +0x705 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc00341a900) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc00341a900) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc00341a900, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [k8s.io] Basic StatefulSet functionality [StatefulSetBasic] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/apps/statefulset.go:114 May 14 21:37:08.453: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=statefulset-8972 describe po ss-0' May 14 21:37:08.613: INFO: stderr: "" May 14 21:37:08.613: INFO: stdout: "Name: ss-0\nNamespace: statefulset-8972\nPriority: 0\nNode: node2/10.10.190.208\nStart Time: Fri, 14 May 2021 21:27:08 +0000\nLabels: baz=blah\n controller-revision-hash=ss-65c7964b94\n foo=bar\n statefulset.kubernetes.io/pod-name=ss-0\nAnnotations: k8s.v1.cni.cncf.io/network-status:\n [{\n \"name\": \"default-cni-network\",\n \"interface\": \"eth0\",\n \"ips\": [\n \"10.244.4.132\"\n ],\n \"mac\": \"5e:65:c0:99:b4:d6\",\n \"default\": true,\n \"dns\": {}\n }]\n k8s.v1.cni.cncf.io/networks-status:\n [{\n \"name\": \"default-cni-network\",\n \"interface\": \"eth0\",\n \"ips\": [\n \"10.244.4.132\"\n ],\n \"mac\": \"5e:65:c0:99:b4:d6\",\n \"default\": true,\n \"dns\": {}\n }]\n kubernetes.io/psp: collectd\nStatus: Pending\nIP: 10.244.4.132\nIPs:\n IP: 10.244.4.132\nControlled By: StatefulSet/ss\nContainers:\n webserver:\n Container ID: \n Image: docker.io/library/httpd:2.4.38-alpine\n Image ID: \n Port: <none>\n Host Port: <none>\n State: Waiting\n Reason: ImagePullBackOff\n Ready: False\n Restart Count: 0\n Readiness: http-get http://:80/index.html delay=0s timeout=1s period=1s #success=1 #failure=1\n Environment: <none>\n Mounts:\n /var/run/secrets/kubernetes.io/serviceaccount from default-token-vtdkl (ro)\nConditions:\n Type Status\n Initialized True \n Ready False \n ContainersReady False \n PodScheduled True \nVolumes:\n default-token-vtdkl:\n Type: Secret (a volume populated by a Secret)\n SecretName: default-token-vtdkl\n Optional: false\nQoS Class: BestEffort\nNode-Selectors: <none>\nTolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s\n node.kubernetes.io/unreachable:NoExecute op=Exists for 300s\nEvents:\n Type Reason Age From Message\n ---- ------ ---- ---- -------\n Normal Scheduled 10m default-scheduler Successfully assigned statefulset-8972/ss-0 to node2\n Normal AddedInterface 9m57s multus Add eth0 [10.244.4.132/24]\n Normal Pulling 8m29s (x4 over 9m57s) kubelet Pulling image \"docker.io/library/httpd:2.4.38-alpine\"\n Warning Failed 8m28s (x4 over 9m55s) kubelet Failed to pull image \"docker.io/library/httpd:2.4.38-alpine\": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit\n Warning Failed 8m28s (x4 over 9m55s) kubelet Error: ErrImagePull\n Warning Failed 7m52s (x7 over 9m54s) kubelet Error: ImagePullBackOff\n Normal BackOff 4m51s (x20 over 9m54s) kubelet Back-off pulling image \"docker.io/library/httpd:2.4.38-alpine\"\n" May 14 21:37:08.614: INFO: Output of kubectl describe ss-0: Name: ss-0 Namespace: statefulset-8972 Priority: 0 Node: node2/10.10.190.208 Start Time: Fri, 14 May 2021 21:27:08 +0000 Labels: baz=blah controller-revision-hash=ss-65c7964b94 foo=bar statefulset.kubernetes.io/pod-name=ss-0 Annotations: k8s.v1.cni.cncf.io/network-status: [{ "name": "default-cni-network", "interface": "eth0", "ips": [ "10.244.4.132" ], "mac": "5e:65:c0:99:b4:d6", "default": true, "dns": {} }] k8s.v1.cni.cncf.io/networks-status: [{ "name": "default-cni-network", "interface": "eth0", "ips": [ "10.244.4.132" ], "mac": "5e:65:c0:99:b4:d6", "default": true, "dns": {} }] kubernetes.io/psp: collectd Status: Pending IP: 10.244.4.132 IPs: IP: 10.244.4.132 Controlled By: StatefulSet/ss Containers: webserver: Container ID: Image: docker.io/library/httpd:2.4.38-alpine Image ID: Port: <none> Host Port: <none> State: Waiting Reason: ImagePullBackOff Ready: False Restart Count: 0 Readiness: http-get http://:80/index.html delay=0s timeout=1s period=1s #success=1 #failure=1 Environment: <none> Mounts: /var/run/secrets/kubernetes.io/serviceaccount from default-token-vtdkl (ro) Conditions: Type Status Initialized True Ready False ContainersReady False PodScheduled True Volumes: default-token-vtdkl: Type: Secret (a volume populated by a Secret) SecretName: default-token-vtdkl Optional: false QoS Class: BestEffort Node-Selectors: <none> Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 10m default-scheduler Successfully assigned statefulset-8972/ss-0 to node2 Normal AddedInterface 9m57s multus Add eth0 [10.244.4.132/24] Normal Pulling 8m29s (x4 over 9m57s) kubelet Pulling image "docker.io/library/httpd:2.4.38-alpine" Warning Failed 8m28s (x4 over 9m55s) kubelet Failed to pull image "docker.io/library/httpd:2.4.38-alpine": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit Warning Failed 8m28s (x4 over 9m55s) kubelet Error: ErrImagePull Warning Failed 7m52s (x7 over 9m54s) kubelet Error: ImagePullBackOff Normal BackOff 4m51s (x20 over 9m54s) kubelet Back-off pulling image "docker.io/library/httpd:2.4.38-alpine" May 14 21:37:08.614: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=statefulset-8972 logs ss-0 --tail=100' May 14 21:37:08.773: INFO: rc: 1 May 14 21:37:08.773: INFO: Last 100 log lines of ss-0: May 14 21:37:08.773: INFO: Deleting all statefulset in ns statefulset-8972 May 14 21:37:08.776: INFO: Scaling statefulset ss to 0 May 14 21:37:18.791: INFO: Waiting for statefulset status.replicas updated to 0 May 14 21:37:18.793: INFO: Deleting statefulset ss [AfterEach] [sig-apps] StatefulSet /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "statefulset-8972". STEP: Found 9 events. May 14 21:37:18.805: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for ss-0: { } Scheduled: Successfully assigned statefulset-8972/ss-0 to node2 May 14 21:37:18.805: INFO: At 2021-05-14 21:27:08 +0000 UTC - event for ss: {statefulset-controller } SuccessfulCreate: create Pod ss-0 in StatefulSet ss successful May 14 21:37:18.805: INFO: At 2021-05-14 21:27:11 +0000 UTC - event for ss-0: {kubelet node2} Pulling: Pulling image "docker.io/library/httpd:2.4.38-alpine" May 14 21:37:18.805: INFO: At 2021-05-14 21:27:11 +0000 UTC - event for ss-0: {multus } AddedInterface: Add eth0 [10.244.4.132/24] May 14 21:37:18.805: INFO: At 2021-05-14 21:27:13 +0000 UTC - event for ss-0: {kubelet node2} Failed: Failed to pull image "docker.io/library/httpd:2.4.38-alpine": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 14 21:37:18.805: INFO: At 2021-05-14 21:27:13 +0000 UTC - event for ss-0: {kubelet node2} Failed: Error: ErrImagePull May 14 21:37:18.805: INFO: At 2021-05-14 21:27:14 +0000 UTC - event for ss-0: {kubelet node2} BackOff: Back-off pulling image "docker.io/library/httpd:2.4.38-alpine" May 14 21:37:18.805: INFO: At 2021-05-14 21:27:14 +0000 UTC - event for ss-0: {kubelet node2} Failed: Error: ImagePullBackOff May 14 21:37:18.805: INFO: At 2021-05-14 21:37:08 +0000 UTC - event for ss: {statefulset-controller } SuccessfulDelete: delete Pod ss-0 in StatefulSet ss successful May 14 21:37:18.807: INFO: POD NODE PHASE GRACE CONDITIONS May 14 21:37:18.807: INFO: May 14 21:37:18.811: INFO: Logging node info for node master1 May 14 21:37:18.813: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 36643 0 2021-05-14 19:56:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-14 21:37:08 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-14 21:37:08 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-14 21:37:08 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-14 21:37:08 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 14 21:37:18.814: INFO: Logging kubelet events for node master1 May 14 21:37:18.816: INFO: Logging pods the kubelet thinks is on node master1 May 14 21:37:18.825: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 14 21:37:18.825: INFO: Container kube-proxy ready: true, restart count 2 May 14 21:37:18.825: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 14 21:37:18.825: INFO: Container kube-multus ready: true, restart count 1 May 14 21:37:18.825: INFO: coredns-7677f9bb54-mlcmv started at 2021-05-14 19:59:27 +0000 UTC (0+1 container statuses recorded) May 14 21:37:18.825: INFO: Container coredns ready: true, restart count 1 May 14 21:37:18.825: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 14 21:37:18.825: INFO: Container docker-registry ready: true, restart count 0 May 14 21:37:18.825: INFO: Container nginx ready: true, restart count 0 May 14 21:37:18.825: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 14 21:37:18.825: INFO: Container nfd-controller ready: true, restart count 0 May 14 21:37:18.825: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 14 21:37:18.825: INFO: Container kube-apiserver ready: true, restart count 0 May 14 21:37:18.825: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 14 21:37:18.825: INFO: Init container install-cni ready: true, restart count 0 May 14 21:37:18.825: INFO: Container kube-flannel ready: true, restart count 1 May 14 21:37:18.825: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 14 21:37:18.825: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 14 21:37:18.825: INFO: Container node-exporter ready: true, restart count 0 May 14 21:37:18.825: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 14 21:37:18.825: INFO: Container kube-scheduler ready: true, restart count 0 May 14 21:37:18.826: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 14 21:37:18.826: INFO: Container kube-controller-manager ready: true, restart count 2 W0514 21:37:18.839095 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 14 21:37:18.862: INFO: Latency metrics for node master1 May 14 21:37:18.862: INFO: Logging node info for node master2 May 14 21:37:18.864: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 36854 0 2021-05-14 19:57:04 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-14 21:37:18 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-14 21:37:18 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-14 21:37:18 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-14 21:37:18 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 14 21:37:18.865: INFO: Logging kubelet events for node master2 May 14 21:37:18.867: INFO: Logging pods the kubelet thinks is on node master2 May 14 21:37:18.875: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 14 21:37:18.875: INFO: Container autoscaler ready: true, restart count 2 May 14 21:37:18.875: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 14 21:37:18.875: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 14 21:37:18.875: INFO: Container node-exporter ready: true, restart count 0 May 14 21:37:18.875: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 14 21:37:18.875: INFO: Container kube-apiserver ready: true, restart count 0 May 14 21:37:18.875: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 14 21:37:18.875: INFO: Container kube-controller-manager ready: true, restart count 2 May 14 21:37:18.875: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 14 21:37:18.875: INFO: Container kube-scheduler ready: true, restart count 2 May 14 21:37:18.875: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 14 21:37:18.875: INFO: Container kube-proxy ready: true, restart count 2 May 14 21:37:18.875: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 14 21:37:18.875: INFO: Init container install-cni ready: true, restart count 0 May 14 21:37:18.875: INFO: Container kube-flannel ready: true, restart count 1 May 14 21:37:18.875: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 14 21:37:18.875: INFO: Container kube-multus ready: true, restart count 1 W0514 21:37:18.887993 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 14 21:37:18.911: INFO: Latency metrics for node master2 May 14 21:37:18.911: INFO: Logging node info for node master3 May 14 21:37:18.915: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 36745 0 2021-05-14 19:57:14 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-14 21:37:15 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-14 21:37:15 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-14 21:37:15 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-14 21:37:15 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 14 21:37:18.915: INFO: Logging kubelet events for node master3 May 14 21:37:18.917: INFO: Logging pods the kubelet thinks is on node master3 May 14 21:37:18.925: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 14 21:37:18.925: INFO: Container kube-scheduler ready: true, restart count 3 May 14 21:37:18.925: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 14 21:37:18.925: INFO: Container kube-apiserver ready: true, restart count 0 May 14 21:37:18.925: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 14 21:37:18.925: INFO: Container kube-proxy ready: true, restart count 1 May 14 21:37:18.925: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 14 21:37:18.925: INFO: Init container install-cni ready: true, restart count 0 May 14 21:37:18.925: INFO: Container kube-flannel ready: true, restart count 2 May 14 21:37:18.925: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 14 21:37:18.925: INFO: Container kube-multus ready: true, restart count 1 May 14 21:37:18.925: INFO: coredns-7677f9bb54-qhzrv started at 2021-05-14 19:59:32 +0000 UTC (0+1 container statuses recorded) May 14 21:37:18.925: INFO: Container coredns ready: true, restart count 2 May 14 21:37:18.925: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 14 21:37:18.925: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 14 21:37:18.925: INFO: Container node-exporter ready: true, restart count 0 May 14 21:37:18.925: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 14 21:37:18.925: INFO: Container kube-controller-manager ready: true, restart count 3 W0514 21:37:18.937447 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 14 21:37:18.961: INFO: Latency metrics for node master3 May 14 21:37:18.961: INFO: Logging node info for node node1 May 14 21:37:18.964: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 36645 0 2021-05-14 19:58:21 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-05-14 19:58:21 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-05-14 20:08:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269633024 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884632576 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-14 21:37:09 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-14 21:37:09 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-14 21:37:09 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-14 21:37:09 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 14 21:37:18.965: INFO: Logging kubelet events for node node1 May 14 21:37:18.967: INFO: Logging pods the kubelet thinks is on node node1 May 14 21:37:18.981: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 14 21:37:18.981: INFO: Init container install-cni ready: true, restart count 0 May 14 21:37:18.981: INFO: Container kube-flannel ready: true, restart count 1 May 14 21:37:18.981: INFO: prometheus-operator-5bb8cb9d8f-4lq8j started at 2021-05-14 20:09:49 +0000 UTC (0+2 container statuses recorded) May 14 21:37:18.981: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 14 21:37:18.981: INFO: Container prometheus-operator ready: true, restart count 0 May 14 21:37:18.981: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-bw262 started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 14 21:37:18.981: INFO: Container kube-sriovdp ready: true, restart count 0 May 14 21:37:18.981: INFO: collectd-qhp77 started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 14 21:37:18.981: INFO: Container collectd ready: true, restart count 0 May 14 21:37:18.981: INFO: Container collectd-exporter ready: true, restart count 0 May 14 21:37:18.981: INFO: Container rbac-proxy ready: true, restart count 0 May 14 21:37:18.981: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 14 21:37:18.981: INFO: Container kube-proxy ready: true, restart count 2 May 14 21:37:18.981: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 14 21:37:18.981: INFO: Container kube-multus ready: true, restart count 1 May 14 21:37:18.981: INFO: netserver-0 started at 2021-05-14 21:36:57 +0000 UTC (0+1 container statuses recorded) May 14 21:37:18.981: INFO: Container webserver ready: true, restart count 0 May 14 21:37:18.981: INFO: host-test-container-pod started at 2021-05-14 21:37:13 +0000 UTC (0+1 container statuses recorded) May 14 21:37:18.981: INFO: Container agnhost-container ready: true, restart count 0 May 14 21:37:18.981: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 14 21:37:18.981: INFO: Container nginx-proxy ready: true, restart count 2 May 14 21:37:18.981: INFO: cmk-init-discover-node1-g6c6q started at 2021-05-14 20:08:21 +0000 UTC (0+3 container statuses recorded) May 14 21:37:18.981: INFO: Container discover ready: false, restart count 0 May 14 21:37:18.981: INFO: Container init ready: false, restart count 0 May 14 21:37:18.981: INFO: Container install ready: false, restart count 0 May 14 21:37:18.981: INFO: cmk-9nqjc started at 2021-05-14 20:09:03 +0000 UTC (0+2 container statuses recorded) May 14 21:37:18.981: INFO: Container nodereport ready: true, restart count 0 May 14 21:37:18.981: INFO: Container reconcile ready: true, restart count 0 May 14 21:37:18.981: INFO: node-exporter-r62rn started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 14 21:37:18.981: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 14 21:37:18.981: INFO: Container node-exporter ready: true, restart count 0 May 14 21:37:18.981: INFO: prometheus-k8s-0 started at 2021-05-14 20:09:58 +0000 UTC (0+5 container statuses recorded) May 14 21:37:18.981: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 14 21:37:18.981: INFO: Container grafana ready: true, restart count 0 May 14 21:37:18.981: INFO: Container prometheus ready: true, restart count 1 May 14 21:37:18.981: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 14 21:37:18.981: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 14 21:37:18.981: INFO: fail-once-local-frcf7 started at 2021-05-14 21:26:40 +0000 UTC (0+1 container statuses recorded) May 14 21:37:18.981: INFO: Container c ready: false, restart count 0 May 14 21:37:18.981: INFO: affinity-nodeport-transition-lczlc started at 2021-05-14 21:35:18 +0000 UTC (0+1 container statuses recorded) May 14 21:37:18.981: INFO: Container affinity-nodeport-transition ready: true, restart count 0 May 14 21:37:18.981: INFO: test-container-pod started at 2021-05-14 21:37:13 +0000 UTC (0+1 container statuses recorded) May 14 21:37:18.981: INFO: Container webserver ready: true, restart count 0 May 14 21:37:18.981: INFO: kubernetes-metrics-scraper-678c97765c-pqnd6 started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 14 21:37:18.981: INFO: Container kubernetes-metrics-scraper ready: true, restart count 1 May 14 21:37:18.981: INFO: node-feature-discovery-worker-nvg9t started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 14 21:37:18.981: INFO: Container nfd-worker ready: true, restart count 0 May 14 21:37:18.981: INFO: pod-b9b8ab41-c0b9-4c1b-ae6a-bf2c755a6cf6 started at 2021-05-14 21:37:15 +0000 UTC (0+1 container statuses recorded) May 14 21:37:18.982: INFO: Container test-container ready: false, restart count 0 W0514 21:37:18.995146 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 14 21:37:19.048: INFO: Latency metrics for node node1 May 14 21:37:19.048: INFO: Logging node info for node node2 May 14 21:37:19.051: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 36659 0 2021-05-14 19:58:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-05-14 20:08:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269633024 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884632576 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-14 21:37:10 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-14 21:37:10 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-14 21:37:10 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-14 21:37:10 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 14 21:37:19.052: INFO: Logging kubelet events for node node2 May 14 21:37:19.054: INFO: Logging pods the kubelet thinks is on node node2 May 14 21:37:19.069: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 14 21:37:19.069: INFO: Container cmk-webhook ready: true, restart count 0 May 14 21:37:19.069: INFO: busybox-privileged-false-76610a26-a824-43a7-88c8-818b09512eda started at 2021-05-14 21:34:51 +0000 UTC (0+1 container statuses recorded) May 14 21:37:19.069: INFO: Container busybox-privileged-false-76610a26-a824-43a7-88c8-818b09512eda ready: false, restart count 0 May 14 21:37:19.069: INFO: ss-0 started at 2021-05-14 21:29:44 +0000 UTC (0+1 container statuses recorded) May 14 21:37:19.069: INFO: Container webserver ready: false, restart count 0 May 14 21:37:19.069: INFO: pod-init-821bd929-f761-4afd-8339-08d366c5fb03 started at 2021-05-14 21:35:31 +0000 UTC (2+1 container statuses recorded) May 14 21:37:19.069: INFO: Init container init1 ready: false, restart count 0 May 14 21:37:19.069: INFO: Init container init2 ready: false, restart count 0 May 14 21:37:19.069: INFO: Container run1 ready: false, restart count 0 May 14 21:37:19.069: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 14 21:37:19.069: INFO: Container discover ready: false, restart count 0 May 14 21:37:19.069: INFO: Container init ready: false, restart count 0 May 14 21:37:19.069: INFO: Container install ready: false, restart count 0 May 14 21:37:19.069: INFO: fail-once-local-vsfv6 started at 2021-05-14 21:26:40 +0000 UTC (0+1 container statuses recorded) May 14 21:37:19.069: INFO: Container c ready: false, restart count 0 May 14 21:37:19.069: INFO: affinity-nodeport-transition-6wl4c started at 2021-05-14 21:35:18 +0000 UTC (0+1 container statuses recorded) May 14 21:37:19.069: INFO: Container affinity-nodeport-transition ready: true, restart count 0 May 14 21:37:19.069: INFO: affinity-nodeport-transition-25mlm started at 2021-05-14 21:35:18 +0000 UTC (0+1 container statuses recorded) May 14 21:37:19.069: INFO: Container affinity-nodeport-transition ready: true, restart count 0 May 14 21:37:19.069: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 14 21:37:19.069: INFO: Container kube-proxy ready: true, restart count 2 May 14 21:37:19.069: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 14 21:37:19.069: INFO: Container kube-multus ready: true, restart count 1 May 14 21:37:19.069: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 14 21:37:19.069: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 14 21:37:19.069: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 14 21:37:19.069: INFO: Container nginx-proxy ready: true, restart count 2 May 14 21:37:19.069: INFO: liveness-fc9e948a-1a96-49d7-9470-98c892d6f043 started at 2021-05-14 21:35:03 +0000 UTC (0+1 container statuses recorded) May 14 21:37:19.069: INFO: Container liveness ready: false, restart count 4 May 14 21:37:19.069: INFO: foo-sgn27 started at 2021-05-14 21:37:13 +0000 UTC (0+1 container statuses recorded) May 14 21:37:19.069: INFO: Container c ready: false, restart count 0 May 14 21:37:19.069: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 14 21:37:19.069: INFO: Init container install-cni ready: true, restart count 1 May 14 21:37:19.069: INFO: Container kube-flannel ready: true, restart count 4 May 14 21:37:19.069: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 14 21:37:19.069: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 14 21:37:19.069: INFO: Container node-exporter ready: true, restart count 0 May 14 21:37:19.069: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 14 21:37:19.069: INFO: Container tas-controller ready: true, restart count 0 May 14 21:37:19.069: INFO: Container tas-extender ready: true, restart count 0 May 14 21:37:19.069: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 14 21:37:19.069: INFO: Container collectd ready: true, restart count 0 May 14 21:37:19.069: INFO: Container collectd-exporter ready: true, restart count 0 May 14 21:37:19.069: INFO: Container rbac-proxy ready: true, restart count 0 May 14 21:37:19.069: INFO: netserver-1 started at 2021-05-14 21:36:57 +0000 UTC (0+1 container statuses recorded) May 14 21:37:19.069: INFO: Container webserver ready: true, restart count 0 May 14 21:37:19.069: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 14 21:37:19.069: INFO: Container kube-sriovdp ready: true, restart count 0 May 14 21:37:19.069: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 14 21:37:19.069: INFO: Container nfd-worker ready: true, restart count 0 May 14 21:37:19.069: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 14 21:37:19.069: INFO: Container nodereport ready: true, restart count 0 May 14 21:37:19.069: INFO: Container reconcile ready: true, restart count 0 May 14 21:37:19.069: INFO: execpod-affinitygxlrf started at 2021-05-14 21:35:24 +0000 UTC (0+1 container statuses recorded) May 14 21:37:19.069: INFO: Container agnhost-container ready: true, restart count 0 May 14 21:37:19.069: INFO: foo-p7v5c started at 2021-05-14 21:37:13 +0000 UTC (0+1 container statuses recorded) May 14 21:37:19.069: INFO: Container c ready: false, restart count 0 W0514 21:37:19.081104 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 14 21:37:19.509: INFO: Latency metrics for node node2 May 14 21:37:19.509: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "statefulset-8972" for this suite. /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 May 14 21:42:47.772: Timed out after 300.000s. Expected <v1.PodPhase>: Pending to equal <v1.PodPhase>: Succeeded /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/runtime.go:154 [BeforeEach] [k8s.io] Container Runtime /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 14 21:37:47.739: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename container-runtime STEP: Waiting for a default service account to be provisioned in namespace [It] should report termination message [LinuxOnly] if TerminationMessagePath is set as non-root user and at a non-default path [NodeConformance] [Conformance] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 STEP: create the container STEP: wait for the container to reach Succeeded May 14 21:42:47.772: FAIL: Timed out after 300.000s. Expected <v1.PodPhase>: Pending to equal <v1.PodPhase>: Succeeded Full Stack Trace k8s.io/kubernetes/test/e2e/common.glob..func25.1.2.1(0x4c5d8a0, 0x1d, 0xc00092f120, 0x1e, 0xc000c70920, 0x2, 0x2, 0xc000c64620, 0x1, 0x1, ...) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/runtime.go:154 +0x3f1 k8s.io/kubernetes/test/e2e/common.glob..func25.1.2.3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/runtime.go:211 +0x292 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc00341a900) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc00341a900) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc00341a900, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [k8s.io] Container Runtime /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "container-runtime-8889". STEP: Found 10 events. May 14 21:42:47.786: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for termination-message-container687358b8-647e-4e81-9fa0-8457606cb829: { } Scheduled: Successfully assigned container-runtime-8889/termination-message-container687358b8-647e-4e81-9fa0-8457606cb829 to node2 May 14 21:42:47.786: INFO: At 2021-05-14 21:37:49 +0000 UTC - event for termination-message-container687358b8-647e-4e81-9fa0-8457606cb829: {multus } AddedInterface: Add eth0 [10.244.4.199/24] May 14 21:42:47.786: INFO: At 2021-05-14 21:37:49 +0000 UTC - event for termination-message-container687358b8-647e-4e81-9fa0-8457606cb829: {kubelet node2} Pulling: Pulling image "docker.io/library/busybox:1.29" May 14 21:42:47.786: INFO: At 2021-05-14 21:37:50 +0000 UTC - event for termination-message-container687358b8-647e-4e81-9fa0-8457606cb829: {kubelet node2} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 14 21:42:47.786: INFO: At 2021-05-14 21:37:50 +0000 UTC - event for termination-message-container687358b8-647e-4e81-9fa0-8457606cb829: {kubelet node2} Failed: Error: ErrImagePull May 14 21:42:47.786: INFO: At 2021-05-14 21:37:52 +0000 UTC - event for termination-message-container687358b8-647e-4e81-9fa0-8457606cb829: {kubelet node2} SandboxChanged: Pod sandbox changed, it will be killed and re-created. May 14 21:42:47.786: INFO: At 2021-05-14 21:37:59 +0000 UTC - event for termination-message-container687358b8-647e-4e81-9fa0-8457606cb829: {multus } AddedInterface: Add eth0 [10.244.4.205/24] May 14 21:42:47.786: INFO: At 2021-05-14 21:37:59 +0000 UTC - event for termination-message-container687358b8-647e-4e81-9fa0-8457606cb829: {kubelet node2} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 14 21:42:47.786: INFO: At 2021-05-14 21:37:59 +0000 UTC - event for termination-message-container687358b8-647e-4e81-9fa0-8457606cb829: {kubelet node2} Failed: Error: ImagePullBackOff May 14 21:42:47.786: INFO: At 2021-05-14 21:38:01 +0000 UTC - event for termination-message-container687358b8-647e-4e81-9fa0-8457606cb829: {multus } AddedInterface: Add eth0 [10.244.4.206/24] May 14 21:42:47.788: INFO: POD NODE PHASE GRACE CONDITIONS May 14 21:42:47.788: INFO: May 14 21:42:47.792: INFO: Logging node info for node master1 May 14 21:42:47.794: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 40803 0 2021-05-14 19:56:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-14 21:42:40 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-14 21:42:40 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-14 21:42:40 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-14 21:42:40 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 14 21:42:47.795: INFO: Logging kubelet events for node master1 May 14 21:42:47.798: INFO: Logging pods the kubelet thinks is on node master1 May 14 21:42:47.807: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 14 21:42:47.807: INFO: Container kube-controller-manager ready: true, restart count 2 May 14 21:42:47.807: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 14 21:42:47.807: INFO: Init container install-cni ready: true, restart count 0 May 14 21:42:47.807: INFO: Container kube-flannel ready: true, restart count 1 May 14 21:42:47.807: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 14 21:42:47.807: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 14 21:42:47.807: INFO: Container node-exporter ready: true, restart count 0 May 14 21:42:47.807: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 14 21:42:47.807: INFO: Container kube-scheduler ready: true, restart count 0 May 14 21:42:47.807: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 14 21:42:47.807: INFO: Container kube-apiserver ready: true, restart count 0 May 14 21:42:47.807: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 14 21:42:47.807: INFO: Container kube-proxy ready: true, restart count 2 May 14 21:42:47.807: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 14 21:42:47.807: INFO: Container kube-multus ready: true, restart count 1 May 14 21:42:47.807: INFO: coredns-7677f9bb54-mlcmv started at 2021-05-14 19:59:27 +0000 UTC (0+1 container statuses recorded) May 14 21:42:47.807: INFO: Container coredns ready: true, restart count 1 May 14 21:42:47.807: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 14 21:42:47.807: INFO: Container docker-registry ready: true, restart count 0 May 14 21:42:47.807: INFO: Container nginx ready: true, restart count 0 May 14 21:42:47.807: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 14 21:42:47.807: INFO: Container nfd-controller ready: true, restart count 0 W0514 21:42:47.819843 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 14 21:42:47.844: INFO: Latency metrics for node master1 May 14 21:42:47.844: INFO: Logging node info for node master2 May 14 21:42:47.847: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 40797 0 2021-05-14 19:57:04 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-14 21:42:39 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-14 21:42:39 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-14 21:42:39 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-14 21:42:39 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 14 21:42:47.848: INFO: Logging kubelet events for node master2 May 14 21:42:47.850: INFO: Logging pods the kubelet thinks is on node master2 May 14 21:42:47.858: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 14 21:42:47.858: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 14 21:42:47.858: INFO: Container node-exporter ready: true, restart count 0 May 14 21:42:47.858: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 14 21:42:47.858: INFO: Container kube-apiserver ready: true, restart count 0 May 14 21:42:47.858: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 14 21:42:47.858: INFO: Container kube-controller-manager ready: true, restart count 2 May 14 21:42:47.858: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 14 21:42:47.858: INFO: Container kube-scheduler ready: true, restart count 2 May 14 21:42:47.858: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 14 21:42:47.858: INFO: Container kube-proxy ready: true, restart count 2 May 14 21:42:47.858: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 14 21:42:47.858: INFO: Init container install-cni ready: true, restart count 0 May 14 21:42:47.858: INFO: Container kube-flannel ready: true, restart count 1 May 14 21:42:47.858: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 14 21:42:47.858: INFO: Container kube-multus ready: true, restart count 1 May 14 21:42:47.858: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 14 21:42:47.858: INFO: Container autoscaler ready: true, restart count 2 W0514 21:42:47.871238 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 14 21:42:47.901: INFO: Latency metrics for node master2 May 14 21:42:47.901: INFO: Logging node info for node master3 May 14 21:42:47.904: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 40876 0 2021-05-14 19:57:14 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-14 21:42:47 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-14 21:42:47 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-14 21:42:47 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-14 21:42:47 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 14 21:42:47.905: INFO: Logging kubelet events for node master3 May 14 21:42:47.907: INFO: Logging pods the kubelet thinks is on node master3 May 14 21:42:47.915: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 14 21:42:47.915: INFO: Container kube-controller-manager ready: true, restart count 3 May 14 21:42:47.915: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 14 21:42:47.915: INFO: Container kube-scheduler ready: true, restart count 3 May 14 21:42:47.915: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 14 21:42:47.915: INFO: Container kube-apiserver ready: true, restart count 0 May 14 21:42:47.915: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 14 21:42:47.915: INFO: Container kube-proxy ready: true, restart count 1 May 14 21:42:47.915: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 14 21:42:47.915: INFO: Init container install-cni ready: true, restart count 0 May 14 21:42:47.915: INFO: Container kube-flannel ready: true, restart count 2 May 14 21:42:47.915: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 14 21:42:47.916: INFO: Container kube-multus ready: true, restart count 1 May 14 21:42:47.916: INFO: coredns-7677f9bb54-qhzrv started at 2021-05-14 19:59:32 +0000 UTC (0+1 container statuses recorded) May 14 21:42:47.916: INFO: Container coredns ready: true, restart count 2 May 14 21:42:47.916: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 14 21:42:47.916: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 14 21:42:47.916: INFO: Container node-exporter ready: true, restart count 0 W0514 21:42:47.934112 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 14 21:42:47.960: INFO: Latency metrics for node master3 May 14 21:42:47.960: INFO: Logging node info for node node1 May 14 21:42:47.962: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 40813 0 2021-05-14 19:58:21 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-05-14 19:58:21 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-05-14 20:08:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269633024 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884632576 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-14 21:42:41 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-14 21:42:41 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-14 21:42:41 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-14 21:42:41 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 14 21:42:47.963: INFO: Logging kubelet events for node node1 May 14 21:42:47.965: INFO: Logging pods the kubelet thinks is on node node1 May 14 21:42:47.981: INFO: node-feature-discovery-worker-nvg9t started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 14 21:42:47.981: INFO: Container nfd-worker ready: true, restart count 0 May 14 21:42:47.981: INFO: cmk-9nqjc started at 2021-05-14 20:09:03 +0000 UTC (0+2 container statuses recorded) May 14 21:42:47.981: INFO: Container nodereport ready: true, restart count 0 May 14 21:42:47.981: INFO: Container reconcile ready: true, restart count 0 May 14 21:42:47.981: INFO: node-exporter-r62rn started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 14 21:42:47.981: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 14 21:42:47.981: INFO: Container node-exporter ready: true, restart count 0 May 14 21:42:47.981: INFO: prometheus-k8s-0 started at 2021-05-14 20:09:58 +0000 UTC (0+5 container statuses recorded) May 14 21:42:47.981: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 14 21:42:47.981: INFO: Container grafana ready: true, restart count 0 May 14 21:42:47.981: INFO: Container prometheus ready: true, restart count 1 May 14 21:42:47.981: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 14 21:42:47.981: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 14 21:42:47.981: INFO: busybox-readonly-fs95521a0d-8d92-4c39-9efe-8b52e58ebe1f started at 2021-05-14 21:41:45 +0000 UTC (0+1 container statuses recorded) May 14 21:42:47.981: INFO: Container busybox-readonly-fs95521a0d-8d92-4c39-9efe-8b52e58ebe1f ready: false, restart count 0 May 14 21:42:47.981: INFO: kubernetes-metrics-scraper-678c97765c-pqnd6 started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 14 21:42:47.981: INFO: Container kubernetes-metrics-scraper ready: true, restart count 1 May 14 21:42:47.981: INFO: prometheus-operator-5bb8cb9d8f-4lq8j started at 2021-05-14 20:09:49 +0000 UTC (0+2 container statuses recorded) May 14 21:42:47.981: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 14 21:42:47.981: INFO: Container prometheus-operator ready: true, restart count 0 May 14 21:42:47.981: INFO: busybox-readonly-false-12b78e9f-4ce2-4bc9-a9b9-cece12f17e37 started at 2021-05-14 21:40:32 +0000 UTC (0+1 container statuses recorded) May 14 21:42:47.981: INFO: Container busybox-readonly-false-12b78e9f-4ce2-4bc9-a9b9-cece12f17e37 ready: false, restart count 0 May 14 21:42:47.981: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 14 21:42:47.981: INFO: Init container install-cni ready: true, restart count 0 May 14 21:42:47.981: INFO: Container kube-flannel ready: true, restart count 1 May 14 21:42:47.981: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 14 21:42:47.981: INFO: Container kube-multus ready: true, restart count 1 May 14 21:42:47.981: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-bw262 started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 14 21:42:47.981: INFO: Container kube-sriovdp ready: true, restart count 0 May 14 21:42:47.981: INFO: collectd-qhp77 started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 14 21:42:47.981: INFO: Container collectd ready: true, restart count 0 May 14 21:42:47.981: INFO: Container collectd-exporter ready: true, restart count 0 May 14 21:42:47.981: INFO: Container rbac-proxy ready: true, restart count 0 May 14 21:42:47.981: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 14 21:42:47.981: INFO: Container kube-proxy ready: true, restart count 2 May 14 21:42:47.981: INFO: cmk-init-discover-node1-g6c6q started at 2021-05-14 20:08:21 +0000 UTC (0+3 container statuses recorded) May 14 21:42:47.981: INFO: Container discover ready: false, restart count 0 May 14 21:42:47.981: INFO: Container init ready: false, restart count 0 May 14 21:42:47.981: INFO: Container install ready: false, restart count 0 May 14 21:42:47.981: INFO: pod-configmaps-a56d2c67-cef4-48ce-9891-bb9c16eefc40 started at 2021-05-14 21:40:34 +0000 UTC (0+1 container statuses recorded) May 14 21:42:47.981: INFO: Container env-test ready: false, restart count 0 May 14 21:42:47.981: INFO: var-expansion-ed77f241-e337-4a28-b2ef-e2093122ec90 started at 2021-05-14 21:41:55 +0000 UTC (0+1 container statuses recorded) May 14 21:42:47.981: INFO: Container dapi-container ready: false, restart count 0 May 14 21:42:47.981: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 14 21:42:47.981: INFO: Container nginx-proxy ready: true, restart count 2 W0514 21:42:47.994360 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 14 21:42:48.038: INFO: Latency metrics for node node1 May 14 21:42:48.038: INFO: Logging node info for node node2 May 14 21:42:48.040: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 40849 0 2021-05-14 19:58:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-05-14 20:08:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269633024 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884632576 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-14 21:42:44 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-14 21:42:44 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-14 21:42:44 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-14 21:42:44 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 14 21:42:48.041: INFO: Logging kubelet events for node node2 May 14 21:42:48.043: INFO: Logging pods the kubelet thinks is on node node2 May 14 21:42:48.129: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 14 21:42:48.129: INFO: Container nginx-proxy ready: true, restart count 2 May 14 21:42:48.129: INFO: foo-sgn27 started at 2021-05-14 21:37:13 +0000 UTC (0+1 container statuses recorded) May 14 21:42:48.129: INFO: Container c ready: false, restart count 0 May 14 21:42:48.129: INFO: server-envvars-d9350368-6516-4cd8-b4ef-7d121efaa512 started at 2021-05-14 21:39:22 +0000 UTC (0+1 container statuses recorded) May 14 21:42:48.129: INFO: Container srv ready: true, restart count 0 May 14 21:42:48.129: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 14 21:42:48.129: INFO: Init container install-cni ready: true, restart count 1 May 14 21:42:48.129: INFO: Container kube-flannel ready: true, restart count 4 May 14 21:42:48.129: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 14 21:42:48.129: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 14 21:42:48.129: INFO: Container node-exporter ready: true, restart count 0 May 14 21:42:48.129: INFO: var-expansion-e55b542c-ffbb-44db-b637-ad22cd1cd7e2 started at 2021-05-14 21:42:06 +0000 UTC (0+1 container statuses recorded) May 14 21:42:48.129: INFO: Container dapi-container ready: false, restart count 0 May 14 21:42:48.129: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 14 21:42:48.129: INFO: Container tas-controller ready: true, restart count 0 May 14 21:42:48.129: INFO: Container tas-extender ready: true, restart count 0 May 14 21:42:48.129: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 14 21:42:48.129: INFO: Container collectd ready: true, restart count 0 May 14 21:42:48.129: INFO: Container collectd-exporter ready: true, restart count 0 May 14 21:42:48.129: INFO: Container rbac-proxy ready: true, restart count 0 May 14 21:42:48.130: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 14 21:42:48.130: INFO: Container kube-sriovdp ready: true, restart count 0 May 14 21:42:48.130: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 14 21:42:48.130: INFO: Container nfd-worker ready: true, restart count 0 May 14 21:42:48.130: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 14 21:42:48.130: INFO: Container nodereport ready: true, restart count 0 May 14 21:42:48.130: INFO: Container reconcile ready: true, restart count 0 May 14 21:42:48.130: INFO: client-envvars-be50a1fb-b455-4ffc-afa6-853b330836c8 started at 2021-05-14 21:39:26 +0000 UTC (0+1 container statuses recorded) May 14 21:42:48.130: INFO: Container env3cont ready: false, restart count 0 May 14 21:42:48.130: INFO: foo-p7v5c started at 2021-05-14 21:37:13 +0000 UTC (0+1 container statuses recorded) May 14 21:42:48.130: INFO: Container c ready: false, restart count 0 May 14 21:42:48.130: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 14 21:42:48.130: INFO: Container cmk-webhook ready: true, restart count 0 May 14 21:42:48.130: INFO: var-expansion-19c563a1-1d95-4965-85f4-a07dea8b7180 started at 2021-05-14 21:39:02 +0000 UTC (0+1 container statuses recorded) May 14 21:42:48.130: INFO: Container dapi-container ready: false, restart count 0 May 14 21:42:48.130: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 14 21:42:48.130: INFO: Container discover ready: false, restart count 0 May 14 21:42:48.130: INFO: Container init ready: false, restart count 0 May 14 21:42:48.130: INFO: Container install ready: false, restart count 0 May 14 21:42:48.130: INFO: test-host-network-pod started at 2021-05-14 21:42:41 +0000 UTC (0+2 container statuses recorded) May 14 21:42:48.130: INFO: Container busybox-1 ready: true, restart count 0 May 14 21:42:48.130: INFO: Container busybox-2 ready: true, restart count 0 May 14 21:42:48.130: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 14 21:42:48.130: INFO: Container kube-proxy ready: true, restart count 2 May 14 21:42:48.130: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 14 21:42:48.130: INFO: Container kube-multus ready: true, restart count 1 May 14 21:42:48.130: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 14 21:42:48.130: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 14 21:42:48.130: INFO: test-pod started at 2021-05-14 21:42:37 +0000 UTC (0+3 container statuses recorded) May 14 21:42:48.130: INFO: Container busybox-1 ready: true, restart count 0 May 14 21:42:48.130: INFO: Container busybox-2 ready: true, restart count 0 May 14 21:42:48.130: INFO: Container busybox-3 ready: true, restart count 0 W0514 21:42:48.143327 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 14 21:42:48.182: INFO: Latency metrics for node node2 May 14 21:42:48.182: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "container-runtime-8889" for this suite. /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 May 14 22:00:12.232: failed to ensure active pods == parallelism in namespace: job-4516 Unexpected error: <*errors.errorString | 0xc0002ba200>: { s: "timed out waiting for the condition", } timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/apps/job.go:193 [BeforeEach] [sig-apps] Job /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 14 21:45:12.198: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename job STEP: Waiting for a default service account to be provisioned in namespace [It] should adopt matching orphans and release non-matching pods [Conformance] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 STEP: Creating a job STEP: Ensuring active pods == parallelism May 14 22:00:12.232: FAIL: failed to ensure active pods == parallelism in namespace: job-4516 Unexpected error: <*errors.errorString | 0xc0002ba200>: { s: "timed out waiting for the condition", } timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/apps.glob..func6.7() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/apps/job.go:193 +0x39a k8s.io/kubernetes/test/e2e.RunE2ETests(0xc00341a900) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc00341a900) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc00341a900, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [sig-apps] Job /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "job-4516". STEP: Found 16 events. May 14 22:00:12.236: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for adopt-release-j65w5: { } Scheduled: Successfully assigned job-4516/adopt-release-j65w5 to node1 May 14 22:00:12.236: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for adopt-release-kfjkh: { } Scheduled: Successfully assigned job-4516/adopt-release-kfjkh to node2 May 14 22:00:12.236: INFO: At 2021-05-14 21:45:12 +0000 UTC - event for adopt-release: {job-controller } SuccessfulCreate: Created pod: adopt-release-kfjkh May 14 22:00:12.236: INFO: At 2021-05-14 21:45:12 +0000 UTC - event for adopt-release: {job-controller } SuccessfulCreate: Created pod: adopt-release-j65w5 May 14 22:00:12.236: INFO: At 2021-05-14 21:45:13 +0000 UTC - event for adopt-release-j65w5: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 14 22:00:12.236: INFO: At 2021-05-14 21:45:13 +0000 UTC - event for adopt-release-j65w5: {multus } AddedInterface: Add eth0 [10.244.3.184/24] May 14 22:00:12.236: INFO: At 2021-05-14 21:45:13 +0000 UTC - event for adopt-release-kfjkh: {kubelet node2} Pulling: Pulling image "docker.io/library/busybox:1.29" May 14 22:00:12.236: INFO: At 2021-05-14 21:45:13 +0000 UTC - event for adopt-release-kfjkh: {multus } AddedInterface: Add eth0 [10.244.4.238/24] May 14 22:00:12.236: INFO: At 2021-05-14 21:45:14 +0000 UTC - event for adopt-release-j65w5: {kubelet node1} Failed: Error: ErrImagePull May 14 22:00:12.236: INFO: At 2021-05-14 21:45:14 +0000 UTC - event for adopt-release-j65w5: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 14 22:00:12.236: INFO: At 2021-05-14 21:45:14 +0000 UTC - event for adopt-release-kfjkh: {kubelet node2} Failed: Error: ErrImagePull May 14 22:00:12.236: INFO: At 2021-05-14 21:45:14 +0000 UTC - event for adopt-release-kfjkh: {kubelet node2} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 14 22:00:12.236: INFO: At 2021-05-14 21:45:15 +0000 UTC - event for adopt-release-j65w5: {kubelet node1} Failed: Error: ImagePullBackOff May 14 22:00:12.236: INFO: At 2021-05-14 21:45:15 +0000 UTC - event for adopt-release-j65w5: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 14 22:00:12.236: INFO: At 2021-05-14 21:45:15 +0000 UTC - event for adopt-release-kfjkh: {kubelet node2} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 14 22:00:12.236: INFO: At 2021-05-14 21:45:15 +0000 UTC - event for adopt-release-kfjkh: {kubelet node2} Failed: Error: ImagePullBackOff May 14 22:00:12.239: INFO: POD NODE PHASE GRACE CONDITIONS May 14 22:00:12.239: INFO: adopt-release-j65w5 node1 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-14 21:45:12 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-14 21:45:12 +0000 UTC ContainersNotReady containers with unready status: [c]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-14 21:45:12 +0000 UTC ContainersNotReady containers with unready status: [c]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-14 21:45:12 +0000 UTC }] May 14 22:00:12.239: INFO: adopt-release-kfjkh node2 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-14 21:45:12 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-14 21:45:12 +0000 UTC ContainersNotReady containers with unready status: [c]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-14 21:45:12 +0000 UTC ContainersNotReady containers with unready status: [c]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-14 21:45:12 +0000 UTC }] May 14 22:00:12.239: INFO: May 14 22:00:12.242: INFO: Logging node info for node master1 May 14 22:00:12.244: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 8e20012e-a811-456d-9add-2ea316e23700 47693 0 2021-05-14 19:56:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"a6:a3:7b:a0:c9:75"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:56:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:56:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:00 +0000 UTC,LastTransitionTime:2021-05-14 20:02:00 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-14 22:00:04 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-14 22:00:04 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-14 22:00:04 +0000 UTC,LastTransitionTime:2021-05-14 19:56:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-14 22:00:04 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a88b162033bc4931ba0342c7f78a28b9,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:ba5ed4e5-a8ef-4986-946f-e7e2d91395d2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 14 22:00:12.245: INFO: Logging kubelet events for node master1 May 14 22:00:12.248: INFO: Logging pods the kubelet thinks is on node master1 May 14 22:00:12.261: INFO: kube-controller-manager-master1 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.261: INFO: Container kube-controller-manager ready: true, restart count 2 May 14 22:00:12.261: INFO: kube-flannel-cx7s6 started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 14 22:00:12.261: INFO: Init container install-cni ready: true, restart count 0 May 14 22:00:12.261: INFO: Container kube-flannel ready: true, restart count 1 May 14 22:00:12.261: INFO: node-exporter-nvrxr started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 14 22:00:12.261: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 14 22:00:12.261: INFO: Container node-exporter ready: true, restart count 0 May 14 22:00:12.262: INFO: kube-scheduler-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.262: INFO: Container kube-scheduler ready: true, restart count 0 May 14 22:00:12.262: INFO: kube-apiserver-master1 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.262: INFO: Container kube-apiserver ready: true, restart count 0 May 14 22:00:12.262: INFO: kube-proxy-v2c76 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.262: INFO: Container kube-proxy ready: true, restart count 2 May 14 22:00:12.262: INFO: kube-multus-ds-amd64-m54v2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.262: INFO: Container kube-multus ready: true, restart count 1 May 14 22:00:12.262: INFO: coredns-7677f9bb54-mlcmv started at 2021-05-14 19:59:27 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.262: INFO: Container coredns ready: true, restart count 1 May 14 22:00:12.262: INFO: docker-registry-docker-registry-56cbc7bc58-bjc5h started at 2021-05-14 20:02:43 +0000 UTC (0+2 container statuses recorded) May 14 22:00:12.262: INFO: Container docker-registry ready: true, restart count 0 May 14 22:00:12.262: INFO: Container nginx ready: true, restart count 0 May 14 22:00:12.262: INFO: node-feature-discovery-controller-5bf5c49849-27v77 started at 2021-05-14 20:05:52 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.262: INFO: Container nfd-controller ready: true, restart count 0 W0514 22:00:12.273902 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 14 22:00:12.298: INFO: Latency metrics for node master1 May 14 22:00:12.298: INFO: Logging node info for node master2 May 14 22:00:12.300: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e8f1881f-5ded-4c6c-b7e6-eb354b7962e2 47692 0 2021-05-14 19:57:04 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:97:9a:eb:9d:a8"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:02:07 +0000 UTC,LastTransitionTime:2021-05-14 20:02:07 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-14 22:00:04 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-14 22:00:04 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-14 22:00:04 +0000 UTC,LastTransitionTime:2021-05-14 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-14 22:00:04 +0000 UTC,LastTransitionTime:2021-05-14 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14c4cdd0613041bb923c5f9b84e0bcde,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:9bdca68c-a5fc-48f7-b392-63d2c04d224d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 14 22:00:12.301: INFO: Logging kubelet events for node master2 May 14 22:00:12.303: INFO: Logging pods the kubelet thinks is on node master2 May 14 22:00:12.318: INFO: kube-scheduler-master2 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.318: INFO: Container kube-scheduler ready: true, restart count 2 May 14 22:00:12.318: INFO: kube-proxy-qcgpm started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.318: INFO: Container kube-proxy ready: true, restart count 2 May 14 22:00:12.318: INFO: kube-flannel-fc4sf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 14 22:00:12.318: INFO: Init container install-cni ready: true, restart count 0 May 14 22:00:12.318: INFO: Container kube-flannel ready: true, restart count 1 May 14 22:00:12.318: INFO: kube-multus-ds-amd64-bt5kr started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.318: INFO: Container kube-multus ready: true, restart count 1 May 14 22:00:12.318: INFO: dns-autoscaler-5b7b5c9b6f-fgzqp started at 2021-05-14 19:59:30 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.318: INFO: Container autoscaler ready: true, restart count 2 May 14 22:00:12.318: INFO: node-exporter-gjrtc started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 14 22:00:12.318: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 14 22:00:12.318: INFO: Container node-exporter ready: true, restart count 0 May 14 22:00:12.318: INFO: kube-apiserver-master2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.318: INFO: Container kube-apiserver ready: true, restart count 0 May 14 22:00:12.318: INFO: kube-controller-manager-master2 started at 2021-05-14 20:01:22 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.318: INFO: Container kube-controller-manager ready: true, restart count 2 W0514 22:00:12.330755 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 14 22:00:12.354: INFO: Latency metrics for node master2 May 14 22:00:12.354: INFO: Logging node info for node master3 May 14 22:00:12.357: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 29fd0a5d-1350-4e28-a4cb-b26dd82cd397 47714 0 2021-05-14 19:57:14 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ae:27:37:b7:ad:a5"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-14 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-14 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-14 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:03 +0000 UTC,LastTransitionTime:2021-05-14 20:01:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-14 22:00:11 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-14 22:00:11 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-14 22:00:11 +0000 UTC,LastTransitionTime:2021-05-14 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-14 22:00:11 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f3fe601830d34e59967ed389af552f25,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:56dd60e2-98fe-4d87-81d9-95db820e7426,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 14 22:00:12.357: INFO: Logging kubelet events for node master3 May 14 22:00:12.359: INFO: Logging pods the kubelet thinks is on node master3 May 14 22:00:12.371: INFO: kube-controller-manager-master3 started at 2021-05-14 20:00:41 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.371: INFO: Container kube-controller-manager ready: true, restart count 3 May 14 22:00:12.371: INFO: kube-scheduler-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.371: INFO: Container kube-scheduler ready: true, restart count 3 May 14 22:00:12.371: INFO: kube-apiserver-master3 started at 2021-05-14 19:57:39 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.371: INFO: Container kube-apiserver ready: true, restart count 0 May 14 22:00:12.371: INFO: kube-proxy-2crs2 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.371: INFO: Container kube-proxy ready: true, restart count 1 May 14 22:00:12.371: INFO: kube-flannel-cl8jf started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 14 22:00:12.371: INFO: Init container install-cni ready: true, restart count 0 May 14 22:00:12.371: INFO: Container kube-flannel ready: true, restart count 2 May 14 22:00:12.371: INFO: kube-multus-ds-amd64-hp6bp started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.371: INFO: Container kube-multus ready: true, restart count 1 May 14 22:00:12.371: INFO: coredns-7677f9bb54-qhzrv started at 2021-05-14 19:59:32 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.371: INFO: Container coredns ready: true, restart count 2 May 14 22:00:12.371: INFO: node-exporter-4cgbq started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 14 22:00:12.371: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 14 22:00:12.371: INFO: Container node-exporter ready: true, restart count 0 W0514 22:00:12.384667 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 14 22:00:12.407: INFO: Latency metrics for node master3 May 14 22:00:12.407: INFO: Logging node info for node node1 May 14 22:00:12.410: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 5e4c0fba-b5fa-4177-b834-f3e04c846ff3 47700 0 2021-05-14 19:58:21 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"ba:ee:c6:a6:52:03"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-05-14 19:58:21 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-05-14 20:08:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269633024 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884632576 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:00:44 +0000 UTC,LastTransitionTime:2021-05-14 20:00:44 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-14 22:00:07 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-14 22:00:07 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-14 22:00:07 +0000 UTC,LastTransitionTime:2021-05-14 19:58:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-14 22:00:07 +0000 UTC,LastTransitionTime:2021-05-14 20:00:41 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:4b96d01fdbcb4fadb4a59fca2e1bf368,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:93c238b3-1895-423c-a1aa-193fbcf8b55f,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:1002487751,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:1636899c10870ab66c48d960a9df620f4f9e86a0c72fbacf36032d27404e7e6c golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 14 22:00:12.411: INFO: Logging kubelet events for node node1 May 14 22:00:12.413: INFO: Logging pods the kubelet thinks is on node node1 May 14 22:00:12.431: INFO: nginx-proxy-node1 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.431: INFO: Container nginx-proxy ready: true, restart count 2 May 14 22:00:12.431: INFO: cmk-init-discover-node1-g6c6q started at 2021-05-14 20:08:21 +0000 UTC (0+3 container statuses recorded) May 14 22:00:12.431: INFO: Container discover ready: false, restart count 0 May 14 22:00:12.431: INFO: Container init ready: false, restart count 0 May 14 22:00:12.431: INFO: Container install ready: false, restart count 0 May 14 22:00:12.431: INFO: prometheus-k8s-0 started at 2021-05-14 20:09:58 +0000 UTC (0+5 container statuses recorded) May 14 22:00:12.431: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 14 22:00:12.431: INFO: Container grafana ready: true, restart count 0 May 14 22:00:12.431: INFO: Container prometheus ready: true, restart count 1 May 14 22:00:12.431: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 14 22:00:12.431: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 14 22:00:12.431: INFO: kubernetes-metrics-scraper-678c97765c-pqnd6 started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.431: INFO: Container kubernetes-metrics-scraper ready: true, restart count 1 May 14 22:00:12.431: INFO: node-feature-discovery-worker-nvg9t started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.431: INFO: Container nfd-worker ready: true, restart count 0 May 14 22:00:12.431: INFO: cmk-9nqjc started at 2021-05-14 20:09:03 +0000 UTC (0+2 container statuses recorded) May 14 22:00:12.431: INFO: Container nodereport ready: true, restart count 0 May 14 22:00:12.431: INFO: Container reconcile ready: true, restart count 0 May 14 22:00:12.431: INFO: node-exporter-r62rn started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 14 22:00:12.431: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 14 22:00:12.431: INFO: Container node-exporter ready: true, restart count 0 May 14 22:00:12.432: INFO: adopt-release-j65w5 started at 2021-05-14 21:45:12 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.432: INFO: Container c ready: false, restart count 0 May 14 22:00:12.432: INFO: kube-flannel-hj8sj started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 14 22:00:12.432: INFO: Init container install-cni ready: true, restart count 0 May 14 22:00:12.432: INFO: Container kube-flannel ready: true, restart count 1 May 14 22:00:12.432: INFO: prometheus-operator-5bb8cb9d8f-4lq8j started at 2021-05-14 20:09:49 +0000 UTC (0+2 container statuses recorded) May 14 22:00:12.432: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 14 22:00:12.432: INFO: Container prometheus-operator ready: true, restart count 0 May 14 22:00:12.432: INFO: kube-proxy-l7697 started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.432: INFO: Container kube-proxy ready: true, restart count 2 May 14 22:00:12.432: INFO: kube-multus-ds-amd64-jhf4c started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.432: INFO: Container kube-multus ready: true, restart count 1 May 14 22:00:12.432: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-bw262 started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.432: INFO: Container kube-sriovdp ready: true, restart count 0 May 14 22:00:12.432: INFO: collectd-qhp77 started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 14 22:00:12.432: INFO: Container collectd ready: true, restart count 0 May 14 22:00:12.432: INFO: Container collectd-exporter ready: true, restart count 0 May 14 22:00:12.432: INFO: Container rbac-proxy ready: true, restart count 0 W0514 22:00:12.444079 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 14 22:00:12.476: INFO: Latency metrics for node node1 May 14 22:00:12.476: INFO: Logging node info for node node2 May 14 22:00:12.478: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 0bae98dc-2ebc-4849-b99e-7780a3bea71e 47705 0 2021-05-14 19:58:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"c6:18:ed:95:bb:1a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubeadm Update v1 2021-05-14 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-14 19:59:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-14 20:06:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-14 20:08:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-05-14 20:08:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269633024 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884632576 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-14 20:01:27 +0000 UTC,LastTransitionTime:2021-05-14 20:01:27 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-14 22:00:09 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-14 22:00:09 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-14 22:00:09 +0000 UTC,LastTransitionTime:2021-05-14 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-14 22:00:09 +0000 UTC,LastTransitionTime:2021-05-14 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a3f22fbf9e534ba1819f7a549414a8a6,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:166b6e45-ba8b-4b89-80b0-befc9a0152b8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:869f7b3516c269b43448f1227c57d536e8a4cf723eeef3b5f8b8e224ecbcfd8e localhost:30500/barometer-collectd:stable],SizeBytes:1464261626,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[localhost:30500/cmk@sha256:f759b012c29126f880575ac543d09301d45f0b2b9d0f5329849ea40e65017dde localhost:30500/cmk:v1.5.1],SizeBytes:726663003,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:948a93bc3803d61dd66ab49f99d4cc657e87273682aec7dd5955a000fd17a7e5 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392645,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:e3157cfba16d361ffec06306dd0154c7dca1931cbc4569e3c5822e30e311948b localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:a43c7fdd150533238a300ad84ac906e551111f9b57273afcb8781ee675fd23b3 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 14 22:00:12.479: INFO: Logging kubelet events for node node2 May 14 22:00:12.480: INFO: Logging pods the kubelet thinks is on node node2 May 14 22:00:12.496: INFO: kube-proxy-t524z started at 2021-05-14 19:58:24 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.497: INFO: Container kube-proxy ready: true, restart count 2 May 14 22:00:12.497: INFO: kube-multus-ds-amd64-n7cb2 started at 2021-05-14 19:59:07 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.497: INFO: Container kube-multus ready: true, restart count 1 May 14 22:00:12.497: INFO: kubernetes-dashboard-86c6f9df5b-ndntg started at 2021-05-14 19:59:31 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.497: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 14 22:00:12.497: INFO: nginx-proxy-node2 started at 2021-05-14 20:05:10 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.497: INFO: Container nginx-proxy ready: true, restart count 2 May 14 22:00:12.497: INFO: kube-flannel-rqcwp started at 2021-05-14 19:58:58 +0000 UTC (1+1 container statuses recorded) May 14 22:00:12.497: INFO: Init container install-cni ready: true, restart count 1 May 14 22:00:12.497: INFO: Container kube-flannel ready: true, restart count 4 May 14 22:00:12.497: INFO: node-exporter-rnd5f started at 2021-05-14 20:09:56 +0000 UTC (0+2 container statuses recorded) May 14 22:00:12.497: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 14 22:00:12.497: INFO: Container node-exporter ready: true, restart count 0 May 14 22:00:12.497: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-l5hlq started at 2021-05-14 20:12:48 +0000 UTC (0+2 container statuses recorded) May 14 22:00:12.497: INFO: Container tas-controller ready: true, restart count 0 May 14 22:00:12.497: INFO: Container tas-extender ready: true, restart count 0 May 14 22:00:12.497: INFO: collectd-xzrgs started at 2021-05-14 20:15:36 +0000 UTC (0+3 container statuses recorded) May 14 22:00:12.497: INFO: Container collectd ready: true, restart count 0 May 14 22:00:12.497: INFO: Container collectd-exporter ready: true, restart count 0 May 14 22:00:12.497: INFO: Container rbac-proxy ready: true, restart count 0 May 14 22:00:12.497: INFO: adopt-release-kfjkh started at 2021-05-14 21:45:12 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.497: INFO: Container c ready: false, restart count 0 May 14 22:00:12.497: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-2c2pw started at 2021-05-14 20:06:38 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.497: INFO: Container kube-sriovdp ready: true, restart count 0 May 14 22:00:12.497: INFO: ss2-0 started at 2021-05-14 21:50:46 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.497: INFO: Container webserver ready: false, restart count 0 May 14 22:00:12.497: INFO: node-feature-discovery-worker-76m6b started at 2021-05-14 20:05:42 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.497: INFO: Container nfd-worker ready: true, restart count 0 May 14 22:00:12.497: INFO: cmk-d2qwf started at 2021-05-14 20:09:04 +0000 UTC (0+2 container statuses recorded) May 14 22:00:12.497: INFO: Container nodereport ready: true, restart count 0 May 14 22:00:12.497: INFO: Container reconcile ready: true, restart count 0 May 14 22:00:12.497: INFO: cmk-webhook-6c9d5f8578-pjgxh started at 2021-05-14 20:09:04 +0000 UTC (0+1 container statuses recorded) May 14 22:00:12.497: INFO: Container cmk-webhook ready: true, restart count 0 May 14 22:00:12.497: INFO: cmk-init-discover-node2-j75ff started at 2021-05-14 20:08:41 +0000 UTC (0+3 container statuses recorded) May 14 22:00:12.497: INFO: Container discover ready: false, restart count 0 May 14 22:00:12.497: INFO: Container init ready: false, restart count 0 May 14 22:00:12.497: INFO: Container install ready: false, restart count 0 W0514 22:00:12.507713 31 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 14 22:00:12.563: INFO: Latency metrics for node node2 May 14 22:00:12.563: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "job-4516" for this suite.