2020-11-11 15:16:05,464 - xtesting.ci.run_tests - INFO - Deployment description: +-------------------------+------------------------------------------------------------+ | ENV VAR | VALUE | +-------------------------+------------------------------------------------------------+ | CI_LOOP | daily | | DEBUG | false | | DEPLOY_SCENARIO | k8-nosdn-nofeature-noha | | INSTALLER_TYPE | unknown | | BUILD_TAG | 838ef6b7 | | NODE_NAME | ericsson-pod2 | | TEST_DB_URL | http://testresults.opnfv.org/test/api/v1/results | | TEST_DB_EXT_URL | http://testresults.opnfv.org/test/api/v1/results | | S3_ENDPOINT_URL | https://storage.googleapis.com | | S3_DST_URL | s3://artifacts.opnfv.org/kuberef/838ef6b7/kube_ben | | | ch_node-842929735 | | HTTP_DST_URL | http://artifacts.opnfv.org/kuberef/838ef6b7/kube_b | | | ench_node-842929735 | +-------------------------+------------------------------------------------------------+ 2020-11-11 15:16:05,476 - xtesting.ci.run_tests - INFO - Loading test case 'kube_bench_node'... 2020-11-11 15:16:05,895 - xtesting.ci.run_tests - INFO - Running test case 'kube_bench_node'... 2020-11-11 15:16:05,948 - functest_kubernetes.security.security - INFO - Job kube-bench-node created 2020-11-11 15:16:19,063 - functest_kubernetes.security.security - INFO - kube-bench-node started in 13.17 sec 2020-11-11 15:16:19,091 - functest_kubernetes.security.security - INFO - [{'id': '4', 'version': '1.5', 'text': 'Worker Node Security Configuration', 'node_type': 'node', 'tests': [{'section': '4.1', 'pass': 7, 'fail': 2, 'warn': 1, 'info': 0, 'desc': 'Worker Node Configuration Files', 'results': [{'test_number': '4.1.1', 'test_desc': 'Ensure that the kubelet service file permissions are set to 644 or more restrictive (Scored)', 'audit': "/bin/sh -c 'if test -e /etc/systemd/system/kubelet.service; then stat -c permissions=%a /etc/systemd/system/kubelet.service; fi' ", 'AuditConfig': '', 'type': '', 'remediation': 'Run the below command (based on the file location on your system) on the each worker node.\nFor example,\nchmod 644 /etc/systemd/system/kubelet.service\n', 'test_info': ['Run the below command (based on the file location on your system) on the each worker node.\nFor example,\nchmod 644 /etc/systemd/system/kubelet.service\n'], 'status': 'PASS', 'actual_value': 'permissions=644\n', 'scored': True, 'expected_result': "bitmask '644' AND '644'"}, {'test_number': '4.1.2', 'test_desc': 'Ensure that the kubelet service file ownership is set to root:root (Scored)', 'audit': "/bin/sh -c 'if test -e /etc/systemd/system/kubelet.service; then stat -c %U:%G /etc/systemd/system/kubelet.service; fi' ", 'AuditConfig': '', 'type': '', 'remediation': 'Run the below command (based on the file location on your system) on the each worker node.\nFor example,\nchown root:root /etc/systemd/system/kubelet.service\n', 'test_info': ['Run the below command (based on the file location on your system) on the each worker node.\nFor example,\nchown root:root /etc/systemd/system/kubelet.service\n'], 'status': 'PASS', 'actual_value': 'root:root\n', 'scored': True, 'expected_result': "'root:root' is present"}, {'test_number': '4.1.3', 'test_desc': 'Ensure that the proxy kubeconfig file permissions are set to 644 or more restrictive (Scored)', 'audit': "/bin/sh -c 'if test -e /etc/kubernetes/proxy.conf; then stat -c permissions=%a /etc/kubernetes/proxy.conf; fi' ", 'AuditConfig': '', 'type': '', 'remediation': 'Run the below command (based on the file location on your system) on the each worker node.\nFor example,\nchmod 644 /etc/kubernetes/proxy.conf\n', 'test_info': ['Run the below command (based on the file location on your system) on the each worker node.\nFor example,\nchmod 644 /etc/kubernetes/proxy.conf\n'], 'status': 'FAIL', 'actual_value': '', 'scored': True, 'expected_result': ''}, {'test_number': '4.1.4', 'test_desc': 'Ensure that the proxy kubeconfig file ownership is set to root:root (Scored)', 'audit': "/bin/sh -c 'if test -e /etc/kubernetes/proxy.conf; then stat -c %U:%G /etc/kubernetes/proxy.conf; fi' ", 'AuditConfig': '', 'type': '', 'remediation': 'Run the below command (based on the file location on your system) on the each worker node.\nFor example, chown root:root /etc/kubernetes/proxy.conf\n', 'test_info': ['Run the below command (based on the file location on your system) on the each worker node.\nFor example, chown root:root /etc/kubernetes/proxy.conf\n'], 'status': 'FAIL', 'actual_value': '', 'scored': True, 'expected_result': ''}, {'test_number': '4.1.5', 'test_desc': 'Ensure that the kubelet.conf file permissions are set to 644 or more restrictive (Scored)', 'audit': "/bin/sh -c 'if test -e /etc/kubernetes/kubelet.conf; then stat -c permissions=%a /etc/kubernetes/kubelet.conf; fi' ", 'AuditConfig': '', 'type': '', 'remediation': 'Run the below command (based on the file location on your system) on the each worker node.\nFor example,\nchmod 644 /etc/kubernetes/kubelet.conf\n', 'test_info': ['Run the below command (based on the file location on your system) on the each worker node.\nFor example,\nchmod 644 /etc/kubernetes/kubelet.conf\n'], 'status': 'PASS', 'actual_value': 'permissions=600\n', 'scored': True, 'expected_result': "bitmask '600' AND '644'"}, {'test_number': '4.1.6', 'test_desc': 'Ensure that the kubelet.conf file ownership is set to root:root (Scored)', 'audit': "/bin/sh -c 'if test -e /etc/kubernetes/kubelet.conf; then stat -c %U:%G /etc/kubernetes/kubelet.conf; fi' ", 'AuditConfig': '', 'type': '', 'remediation': 'Run the below command (based on the file location on your system) on the each worker node.\nFor example,\nchown root:root /etc/kubernetes/kubelet.conf\n', 'test_info': ['Run the below command (based on the file location on your system) on the each worker node.\nFor example,\nchown root:root /etc/kubernetes/kubelet.conf\n'], 'status': 'PASS', 'actual_value': 'root:root\n', 'scored': True, 'expected_result': "'root:root' is equal to 'root:root'"}, {'test_number': '4.1.7', 'test_desc': 'Ensure that the certificate authorities file permissions are set to 644 or more restrictive (Scored)', 'audit': '', 'AuditConfig': '', 'type': '', 'remediation': 'Run the following command to modify the file permissions of the\n--client-ca-file chmod 644 \n', 'test_info': ['Run the following command to modify the file permissions of the\n--client-ca-file chmod 644 \n'], 'status': 'WARN', 'actual_value': '', 'scored': True, 'expected_result': '', 'reason': 'There are no tests'}, {'test_number': '4.1.8', 'test_desc': 'Ensure that the client certificate authorities file ownership is set to root:root (Scored)', 'audit': "CAFILE=$(ps -ef | grep kubelet | grep -v apiserver | grep -- --client-ca-file= | awk -F '--client-ca-file=' '{print $2}' | awk '{print $1}')\nif [[ -z $CAFILE ]]; then\n CAFILE=/etc/kubernetes/pki/ca.crt\nfi\nif test -e $CAFILE; then stat -c %U:%G $CAFILE; fi\n", 'AuditConfig': '', 'type': '', 'remediation': 'Run the following command to modify the ownership of the --client-ca-file.\nchown root:root \n', 'test_info': ['Run the following command to modify the ownership of the --client-ca-file.\nchown root:root \n'], 'status': 'PASS', 'actual_value': 'root:root\n', 'scored': True, 'expected_result': "'root:root' is equal to 'root:root'"}, {'test_number': '4.1.9', 'test_desc': 'Ensure that the kubelet configuration file has permissions set to 644 or more restrictive (Scored)', 'audit': "/bin/sh -c 'if test -e /var/lib/kubelet/config.yaml; then stat -c permissions=%a /var/lib/kubelet/config.yaml; fi' ", 'AuditConfig': '', 'type': '', 'remediation': 'Run the following command (using the config file location identified in the Audit step)\nchmod 644 /var/lib/kubelet/config.yaml\n', 'test_info': ['Run the following command (using the config file location identified in the Audit step)\nchmod 644 /var/lib/kubelet/config.yaml\n'], 'status': 'PASS', 'actual_value': 'permissions=644\n', 'scored': True, 'expected_result': "bitmask '644' AND '644'"}, {'test_number': '4.1.10', 'test_desc': 'Ensure that the kubelet configuration file ownership is set to root:root (Scored)', 'audit': "/bin/sh -c 'if test -e /var/lib/kubelet/config.yaml; then stat -c %U:%G /var/lib/kubelet/config.yaml; fi' ", 'AuditConfig': '', 'type': '', 'remediation': 'Run the following command (using the config file location identified in the Audit step)\nchown root:root /var/lib/kubelet/config.yaml\n', 'test_info': ['Run the following command (using the config file location identified in the Audit step)\nchown root:root /var/lib/kubelet/config.yaml\n'], 'status': 'PASS', 'actual_value': 'root:root\n', 'scored': True, 'expected_result': "'root:root' is present"}]}, {'section': '4.2', 'pass': 6, 'fail': 4, 'warn': 3, 'info': 0, 'desc': 'Kubelet', 'results': [{'test_number': '4.2.1', 'test_desc': 'Ensure that the --anonymous-auth argument is set to false (Scored)', 'audit': '/bin/ps -fC kubelet', 'AuditConfig': '/bin/cat /var/lib/kubelet/config.yaml', 'type': '', 'remediation': 'If using a Kubelet config file, edit the file to set authentication: anonymous: enabled to\nfalse.\nIf using executable arguments, edit the kubelet service file\n/etc/systemd/system/kubelet.service on each worker node and\nset the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.\n--anonymous-auth=false\nBased on your system, restart the kubelet service. For example:\nsystemctl daemon-reload\nsystemctl restart kubelet.service\n', 'test_info': ['If using a Kubelet config file, edit the file to set authentication: anonymous: enabled to\nfalse.\nIf using executable arguments, edit the kubelet service file\n/etc/systemd/system/kubelet.service on each worker node and\nset the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.\n--anonymous-auth=false\nBased on your system, restart the kubelet service. For example:\nsystemctl daemon-reload\nsystemctl restart kubelet.service\n'], 'status': 'PASS', 'actual_value': 'address: 0.0.0.0\napiVersion: kubelet.config.k8s.io/v1beta1\nauthentication:\n anonymous:\n enabled: false\n webhook:\n cacheTTL: 2m0s\n enabled: true\n x509:\n clientCAFile: /etc/kubernetes/ssl/ca.crt\nauthorization:\n mode: Webhook\n webhook:\n cacheAuthorizedTTL: 5m0s\n cacheUnauthorizedTTL: 30s\ncgroupDriver: cgroupfs\ncgroupsPerQOS: true\nclusterDNS:\n- 169.254.25.10\nclusterDomain: cluster.local\nconfigMapAndSecretChangeDetectionStrategy: Watch\ncontainerLogMaxFiles: 5\ncontainerLogMaxSize: 10Mi\ncontentType: application/vnd.kubernetes.protobuf\ncpuCFSQuota: true\ncpuCFSQuotaPeriod: 100ms\ncpuManagerPolicy: none\ncpuManagerReconcilePeriod: 10s\nenableControllerAttachDetach: true\nenableDebuggingHandlers: true\nenforceNodeAllocatable:\n- pods\neventBurst: 10\neventRecordQPS: 5\nevictionHard:\n imagefs.available: 15%\n memory.available: 100Mi\n nodefs.available: 10%\n nodefs.inodesFree: 5%\nevictionPressureTransitionPeriod: 5m0s\nfailSwapOn: true\nfileCheckFrequency: 20s\nhairpinMode: promiscuous-bridge\nhealthzBindAddress: 127.0.0.1\nhealthzPort: 10248\nhttpCheckFrequency: 20s\nimageGCHighThresholdPercent: 85\nimageGCLowThresholdPercent: 80\nimageMinimumGCAge: 2m0s\niptablesDropBit: 15\niptablesMasqueradeBit: 14\nkind: KubeletConfiguration\nkubeAPIBurst: 10\nkubeAPIQPS: 5\nmakeIPTablesUtilChains: true\nmaxOpenFiles: 1000000\nmaxPods: 110\nnodeLeaseDurationSeconds: 40\nnodeStatusReportFrequency: 1m0s\nnodeStatusUpdateFrequency: 10s\noomScoreAdj: -999\npodPidsLimit: -1\nport: 10250\nregistryBurst: 10\nregistryPullQPS: 5\nresolvConf: /etc/resolv.conf\nrotateCertificates: true\nruntimeRequestTimeout: 2m0s\nserializeImagePulls: true\nstaticPodPath: /etc/kubernetes/manifests\nstreamingConnectionIdleTimeout: 4h0m0s\nsyncFrequency: 1m0s\ntopologyManagerPolicy: none\nvolumeStatsAggPeriod: 1m0s\n', 'scored': True, 'expected_result': "'false' is equal to 'false'"}, {'test_number': '4.2.2', 'test_desc': 'Ensure that the --authorization-mode argument is not set to AlwaysAllow (Scored)', 'audit': '/bin/ps -fC kubelet', 'AuditConfig': '/bin/cat /var/lib/kubelet/config.yaml', 'type': '', 'remediation': 'If using a Kubelet config file, edit the file to set authorization: mode to Webhook. If\nusing executable arguments, edit the kubelet service file\n/etc/systemd/system/kubelet.service on each worker node and\nset the below parameter in KUBELET_AUTHZ_ARGS variable.\n--authorization-mode=Webhook\nBased on your system, restart the kubelet service. For example:\nsystemctl daemon-reload\nsystemctl restart kubelet.service\n', 'test_info': ['If using a Kubelet config file, edit the file to set authorization: mode to Webhook. If\nusing executable arguments, edit the kubelet service file\n/etc/systemd/system/kubelet.service on each worker node and\nset the below parameter in KUBELET_AUTHZ_ARGS variable.\n--authorization-mode=Webhook\nBased on your system, restart the kubelet service. For example:\nsystemctl daemon-reload\nsystemctl restart kubelet.service\n'], 'status': 'PASS', 'actual_value': 'address: 0.0.0.0\napiVersion: kubelet.config.k8s.io/v1beta1\nauthentication:\n anonymous:\n enabled: false\n webhook:\n cacheTTL: 2m0s\n enabled: true\n x509:\n clientCAFile: /etc/kubernetes/ssl/ca.crt\nauthorization:\n mode: Webhook\n webhook:\n cacheAuthorizedTTL: 5m0s\n cacheUnauthorizedTTL: 30s\ncgroupDriver: cgroupfs\ncgroupsPerQOS: true\nclusterDNS:\n- 169.254.25.10\nclusterDomain: cluster.local\nconfigMapAndSecretChangeDetectionStrategy: Watch\ncontainerLogMaxFiles: 5\ncontainerLogMaxSize: 10Mi\ncontentType: application/vnd.kubernetes.protobuf\ncpuCFSQuota: true\ncpuCFSQuotaPeriod: 100ms\ncpuManagerPolicy: none\ncpuManagerReconcilePeriod: 10s\nenableControllerAttachDetach: true\nenableDebuggingHandlers: true\nenforceNodeAllocatable:\n- pods\neventBurst: 10\neventRecordQPS: 5\nevictionHard:\n imagefs.available: 15%\n memory.available: 100Mi\n nodefs.available: 10%\n nodefs.inodesFree: 5%\nevictionPressureTransitionPeriod: 5m0s\nfailSwapOn: true\nfileCheckFrequency: 20s\nhairpinMode: promiscuous-bridge\nhealthzBindAddress: 127.0.0.1\nhealthzPort: 10248\nhttpCheckFrequency: 20s\nimageGCHighThresholdPercent: 85\nimageGCLowThresholdPercent: 80\nimageMinimumGCAge: 2m0s\niptablesDropBit: 15\niptablesMasqueradeBit: 14\nkind: KubeletConfiguration\nkubeAPIBurst: 10\nkubeAPIQPS: 5\nmakeIPTablesUtilChains: true\nmaxOpenFiles: 1000000\nmaxPods: 110\nnodeLeaseDurationSeconds: 40\nnodeStatusReportFrequency: 1m0s\nnodeStatusUpdateFrequency: 10s\noomScoreAdj: -999\npodPidsLimit: -1\nport: 10250\nregistryBurst: 10\nregistryPullQPS: 5\nresolvConf: /etc/resolv.conf\nrotateCertificates: true\nruntimeRequestTimeout: 2m0s\nserializeImagePulls: true\nstaticPodPath: /etc/kubernetes/manifests\nstreamingConnectionIdleTimeout: 4h0m0s\nsyncFrequency: 1m0s\ntopologyManagerPolicy: none\nvolumeStatsAggPeriod: 1m0s\n', 'scored': True, 'expected_result': " 'Webhook' not have 'AlwaysAllow'"}, {'test_number': '4.2.3', 'test_desc': 'Ensure that the --client-ca-file argument is set as appropriate (Scored)', 'audit': '/bin/ps -fC kubelet', 'AuditConfig': '/bin/cat /var/lib/kubelet/config.yaml', 'type': '', 'remediation': 'If using a Kubelet config file, edit the file to set authentication: x509: clientCAFile to\nthe location of the client CA file.\nIf using command line arguments, edit the kubelet service file\n/etc/systemd/system/kubelet.service on each worker node and\nset the below parameter in KUBELET_AUTHZ_ARGS variable.\n--client-ca-file=\nBased on your system, restart the kubelet service. For example:\nsystemctl daemon-reload\nsystemctl restart kubelet.service\n', 'test_info': ['If using a Kubelet config file, edit the file to set authentication: x509: clientCAFile to\nthe location of the client CA file.\nIf using command line arguments, edit the kubelet service file\n/etc/systemd/system/kubelet.service on each worker node and\nset the below parameter in KUBELET_AUTHZ_ARGS variable.\n--client-ca-file=\nBased on your system, restart the kubelet service. For example:\nsystemctl daemon-reload\nsystemctl restart kubelet.service\n'], 'status': 'PASS', 'actual_value': 'address: 0.0.0.0\napiVersion: kubelet.config.k8s.io/v1beta1\nauthentication:\n anonymous:\n enabled: false\n webhook:\n cacheTTL: 2m0s\n enabled: true\n x509:\n clientCAFile: /etc/kubernetes/ssl/ca.crt\nauthorization:\n mode: Webhook\n webhook:\n cacheAuthorizedTTL: 5m0s\n cacheUnauthorizedTTL: 30s\ncgroupDriver: cgroupfs\ncgroupsPerQOS: true\nclusterDNS:\n- 169.254.25.10\nclusterDomain: cluster.local\nconfigMapAndSecretChangeDetectionStrategy: Watch\ncontainerLogMaxFiles: 5\ncontainerLogMaxSize: 10Mi\ncontentType: application/vnd.kubernetes.protobuf\ncpuCFSQuota: true\ncpuCFSQuotaPeriod: 100ms\ncpuManagerPolicy: none\ncpuManagerReconcilePeriod: 10s\nenableControllerAttachDetach: true\nenableDebuggingHandlers: true\nenforceNodeAllocatable:\n- pods\neventBurst: 10\neventRecordQPS: 5\nevictionHard:\n imagefs.available: 15%\n memory.available: 100Mi\n nodefs.available: 10%\n nodefs.inodesFree: 5%\nevictionPressureTransitionPeriod: 5m0s\nfailSwapOn: true\nfileCheckFrequency: 20s\nhairpinMode: promiscuous-bridge\nhealthzBindAddress: 127.0.0.1\nhealthzPort: 10248\nhttpCheckFrequency: 20s\nimageGCHighThresholdPercent: 85\nimageGCLowThresholdPercent: 80\nimageMinimumGCAge: 2m0s\niptablesDropBit: 15\niptablesMasqueradeBit: 14\nkind: KubeletConfiguration\nkubeAPIBurst: 10\nkubeAPIQPS: 5\nmakeIPTablesUtilChains: true\nmaxOpenFiles: 1000000\nmaxPods: 110\nnodeLeaseDurationSeconds: 40\nnodeStatusReportFrequency: 1m0s\nnodeStatusUpdateFrequency: 10s\noomScoreAdj: -999\npodPidsLimit: -1\nport: 10250\nregistryBurst: 10\nregistryPullQPS: 5\nresolvConf: /etc/resolv.conf\nrotateCertificates: true\nruntimeRequestTimeout: 2m0s\nserializeImagePulls: true\nstaticPodPath: /etc/kubernetes/manifests\nstreamingConnectionIdleTimeout: 4h0m0s\nsyncFrequency: 1m0s\ntopologyManagerPolicy: none\nvolumeStatsAggPeriod: 1m0s\n', 'scored': True, 'expected_result': "'' is present"}, {'test_number': '4.2.4', 'test_desc': 'Ensure that the --read-only-port argument is set to 0 (Scored)', 'audit': '/bin/ps -fC kubelet', 'AuditConfig': '/bin/cat /var/lib/kubelet/config.yaml', 'type': '', 'remediation': 'If using a Kubelet config file, edit the file to set readOnlyPort to 0.\nIf using command line arguments, edit the kubelet service file\n/etc/systemd/system/kubelet.service on each worker node and\nset the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.\n--read-only-port=0\nBased on your system, restart the kubelet service. For example:\nsystemctl daemon-reload\nsystemctl restart kubelet.service\n', 'test_info': ['If using a Kubelet config file, edit the file to set readOnlyPort to 0.\nIf using command line arguments, edit the kubelet service file\n/etc/systemd/system/kubelet.service on each worker node and\nset the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.\n--read-only-port=0\nBased on your system, restart the kubelet service. For example:\nsystemctl daemon-reload\nsystemctl restart kubelet.service\n'], 'status': 'FAIL', 'actual_value': '', 'scored': True, 'expected_result': ''}, {'test_number': '4.2.5', 'test_desc': 'Ensure that the --streaming-connection-idle-timeout argument is not set to 0 (Scored)', 'audit': '/bin/ps -fC kubelet', 'AuditConfig': '/bin/cat /var/lib/kubelet/config.yaml', 'type': '', 'remediation': 'If using a Kubelet config file, edit the file to set streamingConnectionIdleTimeout to a\nvalue other than 0.\nIf using command line arguments, edit the kubelet service file\n/etc/systemd/system/kubelet.service on each worker node and\nset the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.\n--streaming-connection-idle-timeout=5m\nBased on your system, restart the kubelet service. For example:\nsystemctl daemon-reload\nsystemctl restart kubelet.service\n', 'test_info': ['If using a Kubelet config file, edit the file to set streamingConnectionIdleTimeout to a\nvalue other than 0.\nIf using command line arguments, edit the kubelet service file\n/etc/systemd/system/kubelet.service on each worker node and\nset the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.\n--streaming-connection-idle-timeout=5m\nBased on your system, restart the kubelet service. For example:\nsystemctl daemon-reload\nsystemctl restart kubelet.service\n'], 'status': 'PASS', 'actual_value': 'UID PID PPID C STIME TTY TIME CMD\nroot 26911 1 2 11:48 ? 00:05:59 /usr/local/bin/kubelet --logtostderr=true --v=2 --node-ip=10.0.20.14 --hostname-override=node2 --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/etc/kubernetes/kubelet-config.yaml --kubeconfig=/etc/kubernetes/kubelet.conf --pod-infra-container-image=gcr.io/google_containers/pause-amd64:3.1 --runtime-cgroups=/systemd/system.slice --feature-gates=TopologyManager=true --topology-manager-policy=none --network-plugin=cni --cni-conf-dir=/etc/cni/net.d --cni-bin-dir=/opt/cni/bin\n', 'scored': True, 'expected_result': "'--streaming-connection-idle-timeout' is present OR '--streaming-connection-idle-timeout' is not present"}, {'test_number': '4.2.6', 'test_desc': 'Ensure that the --protect-kernel-defaults argument is set to true (Scored)', 'audit': '/bin/ps -fC kubelet', 'AuditConfig': '/bin/cat /var/lib/kubelet/config.yaml', 'type': '', 'remediation': 'If using a Kubelet config file, edit the file to set protectKernelDefaults: true.\nIf using command line arguments, edit the kubelet service file\n/etc/systemd/system/kubelet.service on each worker node and\nset the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.\n--protect-kernel-defaults=true\nBased on your system, restart the kubelet service. For example:\nsystemctl daemon-reload\nsystemctl restart kubelet.service\n', 'test_info': ['If using a Kubelet config file, edit the file to set protectKernelDefaults: true.\nIf using command line arguments, edit the kubelet service file\n/etc/systemd/system/kubelet.service on each worker node and\nset the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.\n--protect-kernel-defaults=true\nBased on your system, restart the kubelet service. For example:\nsystemctl daemon-reload\nsystemctl restart kubelet.service\n'], 'status': 'FAIL', 'actual_value': '', 'scored': True, 'expected_result': ''}, {'test_number': '4.2.7', 'test_desc': 'Ensure that the --make-iptables-util-chains argument is set to true (Scored)', 'audit': '/bin/ps -fC kubelet', 'AuditConfig': '/bin/cat /var/lib/kubelet/config.yaml', 'type': '', 'remediation': 'If using a Kubelet config file, edit the file to set makeIPTablesUtilChains: true.\nIf using command line arguments, edit the kubelet service file\n/etc/systemd/system/kubelet.service on each worker node and\nremove the --make-iptables-util-chains argument from the\nKUBELET_SYSTEM_PODS_ARGS variable.\nBased on your system, restart the kubelet service. For example:\nsystemctl daemon-reload\nsystemctl restart kubelet.service\n', 'test_info': ['If using a Kubelet config file, edit the file to set makeIPTablesUtilChains: true.\nIf using command line arguments, edit the kubelet service file\n/etc/systemd/system/kubelet.service on each worker node and\nremove the --make-iptables-util-chains argument from the\nKUBELET_SYSTEM_PODS_ARGS variable.\nBased on your system, restart the kubelet service. For example:\nsystemctl daemon-reload\nsystemctl restart kubelet.service\n'], 'status': 'PASS', 'actual_value': 'UID PID PPID C STIME TTY TIME CMD\nroot 26911 1 2 11:48 ? 00:05:59 /usr/local/bin/kubelet --logtostderr=true --v=2 --node-ip=10.0.20.14 --hostname-override=node2 --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/etc/kubernetes/kubelet-config.yaml --kubeconfig=/etc/kubernetes/kubelet.conf --pod-infra-container-image=gcr.io/google_containers/pause-amd64:3.1 --runtime-cgroups=/systemd/system.slice --feature-gates=TopologyManager=true --topology-manager-policy=none --network-plugin=cni --cni-conf-dir=/etc/cni/net.d --cni-bin-dir=/opt/cni/bin\n', 'scored': True, 'expected_result': "'--make-iptables-util-chains' is present OR '--make-iptables-util-chains' is not present"}, {'test_number': '4.2.8', 'test_desc': 'Ensure that the --hostname-override argument is not set (Not Scored)', 'audit': '/bin/ps -fC kubelet ', 'AuditConfig': '', 'type': '', 'remediation': 'Edit the kubelet service file /etc/systemd/system/kubelet.service\non each worker node and remove the --hostname-override argument from the\nKUBELET_SYSTEM_PODS_ARGS variable.\nBased on your system, restart the kubelet service. For example:\nsystemctl daemon-reload\nsystemctl restart kubelet.service\n', 'test_info': ['Edit the kubelet service file /etc/systemd/system/kubelet.service\non each worker node and remove the --hostname-override argument from the\nKUBELET_SYSTEM_PODS_ARGS variable.\nBased on your system, restart the kubelet service. For example:\nsystemctl daemon-reload\nsystemctl restart kubelet.service\n'], 'status': 'WARN', 'actual_value': '', 'scored': False, 'expected_result': ''}, {'test_number': '4.2.9', 'test_desc': 'Ensure that the --event-qps argument is set to 0 or a level which ensures appropriate event capture (Not Scored)', 'audit': '/bin/ps -fC kubelet', 'AuditConfig': '/bin/cat /var/lib/kubelet/config.yaml', 'type': '', 'remediation': 'If using a Kubelet config file, edit the file to set eventRecordQPS: to an appropriate level.\nIf using command line arguments, edit the kubelet service file\n/etc/systemd/system/kubelet.service on each worker node and\nset the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.\nBased on your system, restart the kubelet service. For example:\nsystemctl daemon-reload\nsystemctl restart kubelet.service\n', 'test_info': ['If using a Kubelet config file, edit the file to set eventRecordQPS: to an appropriate level.\nIf using command line arguments, edit the kubelet service file\n/etc/systemd/system/kubelet.service on each worker node and\nset the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.\nBased on your system, restart the kubelet service. For example:\nsystemctl daemon-reload\nsystemctl restart kubelet.service\n'], 'status': 'WARN', 'actual_value': '', 'scored': False, 'expected_result': ''}, {'test_number': '4.2.10', 'test_desc': 'Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Scored)', 'audit': '/bin/ps -fC kubelet', 'AuditConfig': '/bin/cat /var/lib/kubelet/config.yaml', 'type': '', 'remediation': 'If using a Kubelet config file, edit the file to set tlsCertFile to the location\nof the certificate file to use to identify this Kubelet, and tlsPrivateKeyFile\nto the location of the corresponding private key file.\nIf using command line arguments, edit the kubelet service file\n/etc/systemd/system/kubelet.service on each worker node and\nset the below parameters in KUBELET_CERTIFICATE_ARGS variable.\n--tls-cert-file=\n--tls-private-key-file=\nBased on your system, restart the kubelet service. For example:\nsystemctl daemon-reload\nsystemctl restart kubelet.service\n', 'test_info': ['If using a Kubelet config file, edit the file to set tlsCertFile to the location\nof the certificate file to use to identify this Kubelet, and tlsPrivateKeyFile\nto the location of the corresponding private key file.\nIf using command line arguments, edit the kubelet service file\n/etc/systemd/system/kubelet.service on each worker node and\nset the below parameters in KUBELET_CERTIFICATE_ARGS variable.\n--tls-cert-file=\n--tls-private-key-file=\nBased on your system, restart the kubelet service. For example:\nsystemctl daemon-reload\nsystemctl restart kubelet.service\n'], 'status': 'FAIL', 'actual_value': '', 'scored': True, 'expected_result': ''}, {'test_number': '4.2.11', 'test_desc': 'Ensure that the --rotate-certificates argument is not set to false (Scored)', 'audit': '/bin/ps -fC kubelet', 'AuditConfig': '/bin/cat /var/lib/kubelet/config.yaml', 'type': '', 'remediation': 'If using a Kubelet config file, edit the file to add the line rotateCertificates: true or\nremove it altogether to use the default value.\nIf using command line arguments, edit the kubelet service file\n/etc/systemd/system/kubelet.service on each worker node and\nremove --rotate-certificates=false argument from the KUBELET_CERTIFICATE_ARGS\nvariable.\nBased on your system, restart the kubelet service. For example:\nsystemctl daemon-reload\nsystemctl restart kubelet.service\n', 'test_info': ['If using a Kubelet config file, edit the file to add the line rotateCertificates: true or\nremove it altogether to use the default value.\nIf using command line arguments, edit the kubelet service file\n/etc/systemd/system/kubelet.service on each worker node and\nremove --rotate-certificates=false argument from the KUBELET_CERTIFICATE_ARGS\nvariable.\nBased on your system, restart the kubelet service. For example:\nsystemctl daemon-reload\nsystemctl restart kubelet.service\n'], 'status': 'PASS', 'actual_value': 'UID PID PPID C STIME TTY TIME CMD\nroot 26911 1 2 11:48 ? 00:05:59 /usr/local/bin/kubelet --logtostderr=true --v=2 --node-ip=10.0.20.14 --hostname-override=node2 --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/etc/kubernetes/kubelet-config.yaml --kubeconfig=/etc/kubernetes/kubelet.conf --pod-infra-container-image=gcr.io/google_containers/pause-amd64:3.1 --runtime-cgroups=/systemd/system.slice --feature-gates=TopologyManager=true --topology-manager-policy=none --network-plugin=cni --cni-conf-dir=/etc/cni/net.d --cni-bin-dir=/opt/cni/bin\n', 'scored': True, 'expected_result': "'--rotate-certificates' is present OR '--rotate-certificates' is not present"}, {'test_number': '4.2.12', 'test_desc': 'Ensure that the RotateKubeletServerCertificate argument is set to true (Scored)', 'audit': '/bin/ps -fC kubelet', 'AuditConfig': '/bin/cat /var/lib/kubelet/config.yaml', 'type': '', 'remediation': 'Edit the kubelet service file /etc/systemd/system/kubelet.service\non each worker node and set the below parameter in KUBELET_CERTIFICATE_ARGS variable.\n--feature-gates=RotateKubeletServerCertificate=true\nBased on your system, restart the kubelet service. For example:\nsystemctl daemon-reload\nsystemctl restart kubelet.service\n', 'test_info': ['Edit the kubelet service file /etc/systemd/system/kubelet.service\non each worker node and set the below parameter in KUBELET_CERTIFICATE_ARGS variable.\n--feature-gates=RotateKubeletServerCertificate=true\nBased on your system, restart the kubelet service. For example:\nsystemctl daemon-reload\nsystemctl restart kubelet.service\n'], 'status': 'FAIL', 'actual_value': '', 'scored': True, 'expected_result': ''}, {'test_number': '4.2.13', 'test_desc': 'Ensure that the Kubelet only makes use of Strong Cryptographic Ciphers (Not Scored)', 'audit': '/bin/ps -fC kubelet', 'AuditConfig': '/bin/cat /var/lib/kubelet/config.yaml', 'type': '', 'remediation': 'If using a Kubelet config file, edit the file to set TLSCipherSuites: to\nTLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256\nor to a subset of these values.\nIf using executable arguments, edit the kubelet service file\n/etc/systemd/system/kubelet.service on each worker node and\nset the --tls-cipher-suites parameter as follows, or to a subset of these values.\n--tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256\nBased on your system, restart the kubelet service. For example:\nsystemctl daemon-reload\nsystemctl restart kubelet.service\n', 'test_info': ['If using a Kubelet config file, edit the file to set TLSCipherSuites: to\nTLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256\nor to a subset of these values.\nIf using executable arguments, edit the kubelet service file\n/etc/systemd/system/kubelet.service on each worker node and\nset the --tls-cipher-suites parameter as follows, or to a subset of these values.\n--tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256\nBased on your system, restart the kubelet service. For example:\nsystemctl daemon-reload\nsystemctl restart kubelet.service\n'], 'status': 'WARN', 'actual_value': '', 'scored': False, 'expected_result': ''}]}], 'total_pass': 13, 'total_fail': 6, 'total_warn': 4, 'total_info': 0}] 2020-11-11 15:16:19,098 - functest_kubernetes.security.security - ERROR - Ensure that the proxy kubeconfig file permissions are set to 644 or more restrictive (Scored) Run the below command (based on the file location on your system) on the each worker node. For example, chmod 644 /etc/kubernetes/proxy.conf 2020-11-11 15:16:19,098 - functest_kubernetes.security.security - ERROR - Ensure that the proxy kubeconfig file ownership is set to root:root (Scored) Run the below command (based on the file location on your system) on the each worker node. For example, chown root:root /etc/kubernetes/proxy.conf 2020-11-11 15:16:19,098 - functest_kubernetes.security.security - ERROR - Ensure that the --read-only-port argument is set to 0 (Scored) If using a Kubelet config file, edit the file to set readOnlyPort to 0. If using command line arguments, edit the kubelet service file /etc/systemd/system/kubelet.service on each worker node and set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable. --read-only-port=0 Based on your system, restart the kubelet service. For example: systemctl daemon-reload systemctl restart kubelet.service 2020-11-11 15:16:19,099 - functest_kubernetes.security.security - ERROR - Ensure that the --protect-kernel-defaults argument is set to true (Scored) If using a Kubelet config file, edit the file to set protectKernelDefaults: true. If using command line arguments, edit the kubelet service file /etc/systemd/system/kubelet.service on each worker node and set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable. --protect-kernel-defaults=true Based on your system, restart the kubelet service. For example: systemctl daemon-reload systemctl restart kubelet.service 2020-11-11 15:16:19,099 - functest_kubernetes.security.security - ERROR - Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Scored) If using a Kubelet config file, edit the file to set tlsCertFile to the location of the certificate file to use to identify this Kubelet, and tlsPrivateKeyFile to the location of the corresponding private key file. If using command line arguments, edit the kubelet service file /etc/systemd/system/kubelet.service on each worker node and set the below parameters in KUBELET_CERTIFICATE_ARGS variable. --tls-cert-file= --tls-private-key-file= Based on your system, restart the kubelet service. For example: systemctl daemon-reload systemctl restart kubelet.service 2020-11-11 15:16:19,099 - functest_kubernetes.security.security - ERROR - Ensure that the RotateKubeletServerCertificate argument is set to true (Scored) Edit the kubelet service file /etc/systemd/system/kubelet.service on each worker node and set the below parameter in KUBELET_CERTIFICATE_ARGS variable. --feature-gates=RotateKubeletServerCertificate=true Based on your system, restart the kubelet service. For example: systemctl daemon-reload systemctl restart kubelet.service 2020-11-11 15:16:19,100 - functest_kubernetes.security.security - WARNING - Targets: +-------------------+-----------------+-----------------------------------------+--------------+--------------+--------------+ | NODE_TYPE | VERSION | TEST_DESC | PASS | FAIL | WARN | +-------------------+-----------------+-----------------------------------------+--------------+--------------+--------------+ | node | 1.5 | Worker Node Configuration Files | 7 | 2 | 1 | | node | 1.5 | Kubelet | 6 | 4 | 3 | +-------------------+-----------------+-----------------------------------------+--------------+--------------+--------------+ 2020-11-11 15:16:19,101 - xtesting.ci.run_tests - INFO - Test result: +-------------------------+------------------+------------------+----------------+ | TEST CASE | PROJECT | DURATION | RESULT | +-------------------------+------------------+------------------+----------------+ | kube_bench_node | functest | 00:13 | PASS | +-------------------------+------------------+------------------+----------------+