2020-08-27 09:14:40,426 - xtesting.ci.run_tests - INFO - Deployment description: +-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ENV VAR | VALUE | +-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | TEST_DB_EXT_URL | http://testresults.opnfv.org/test/api/v1/results | | S3_DST_URL | s3://artifacts.opnfv.org/functest-kubernetes/00HUE8C7BMI7/functest-kubernetes-pi-ollivier-functest-kubernetes-security-arm-hunter-kube_bench-run-23 | | S3_ENDPOINT_URL | https://storage.googleapis.com | | DEPLOY_SCENARIO | k8-nosdn-nofeature-noha | | BUILD_TAG | 00HUE8C7BMI7 | | DEBUG | true | | INSTALLER_TYPE | unknown | | CI_LOOP | daily | | TEST_DB_URL | http://testresults.opnfv.org/test/api/v1/results | | HTTP_DST_URL | http://artifacts.opnfv.org/functest-kubernetes/00HUE8C7BMI7/functest-kubernetes-pi-ollivier-functest-kubernetes-security-arm-hunter-kube_bench-run-23 | | NODE_NAME | lf-virtual1-1 | +-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-08-27 09:14:40,483 - xtesting.ci.run_tests - DEBUG - No env file /var/lib/xtesting/conf/env_file found 2020-08-27 09:14:40,484 - xtesting.ci.run_tests - DEBUG - Test args: kube_bench 2020-08-27 09:14:40,535 - xtesting.ci.run_tests - INFO - Loading test case 'kube_bench'... 2020-08-27 09:14:41,855 - xtesting.ci.run_tests - INFO - Running test case 'kube_bench'... 2020-08-27 09:14:42,132 - kubernetes.client.rest - DEBUG - response body: {"kind":"Namespace","apiVersion":"v1","metadata":{"name":"ims-v2c26","generateName":"ims-","selfLink":"/api/v1/namespaces/ims-v2c26","uid":"bda52821-e845-11ea-a485-0242ac120004","resourceVersion":"2586335","creationTimestamp":"2020-08-27T09:14:42Z"},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Active"}} 2020-08-27 09:14:42,143 - functest_kubernetes.security.security - DEBUG - create_namespace: {'api_version': 'v1', 'kind': 'Namespace', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 27, 9, 14, 42, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'ims-', 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': None, 'name': 'ims-v2c26', 'namespace': None, 'owner_references': None, 'resource_version': '2586335', 'self_link': '/api/v1/namespaces/ims-v2c26', 'uid': 'bda52821-e845-11ea-a485-0242ac120004'}, 'spec': {'finalizers': ['kubernetes']}, 'status': {'phase': 'Active'}} 2020-08-27 09:14:42,491 - kubernetes.client.rest - DEBUG - response body: {"kind":"Job","apiVersion":"batch/v1","metadata":{"name":"kube-bench","namespace":"ims-v2c26","selfLink":"/apis/batch/v1/namespaces/ims-v2c26/jobs/kube-bench","uid":"bddb2b11-e845-11ea-a485-0242ac120004","resourceVersion":"2586342","creationTimestamp":"2020-08-27T09:14:42Z","labels":{"app":"kube-bench","controller-uid":"bddb2b11-e845-11ea-a485-0242ac120004","job-name":"kube-bench"}},"spec":{"parallelism":1,"completions":1,"backoffLimit":6,"selector":{"matchLabels":{"controller-uid":"bddb2b11-e845-11ea-a485-0242ac120004"}},"template":{"metadata":{"creationTimestamp":null,"labels":{"app":"kube-bench","controller-uid":"bddb2b11-e845-11ea-a485-0242ac120004","job-name":"kube-bench"}},"spec":{"volumes":[{"name":"var-lib-etcd","hostPath":{"path":"/var/lib/etcd","type":""}},{"name":"var-lib-kubelet","hostPath":{"path":"/var/lib/kubelet","type":""}},{"name":"etc-systemd","hostPath":{"path":"/etc/systemd","type":""}},{"name":"etc-kubernetes","hostPath":{"path":"/etc/kubernetes","type":""}},{"name":"usr-bin","hostPath":{"path":"/usr/bin","type":""}}],"containers":[{"name":"kube-bench","image":"aquasec/kube-bench:0.3.1","command":["kube-bench"],"resources":{},"volumeMounts":[{"name":"var-lib-etcd","readOnly":true,"mountPath":"/var/lib/etcd"},{"name":"var-lib-kubelet","readOnly":true,"mountPath":"/var/lib/kubelet"},{"name":"etc-systemd","readOnly":true,"mountPath":"/etc/systemd"},{"name":"etc-kubernetes","readOnly":true,"mountPath":"/etc/kubernetes"},{"name":"usr-bin","readOnly":true,"mountPath":"/usr/local/mount-from-host/bin"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Never","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","hostPID":true,"securityContext":{},"schedulerName":"default-scheduler"}}},"status":{}} 2020-08-27 09:14:42,513 - functest_kubernetes.security.security - INFO - Job kube-bench created 2020-08-27 09:14:42,545 - functest_kubernetes.security.security - DEBUG - create_namespaced_job: {'api_version': 'batch/v1', 'kind': 'Job', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 27, 9, 14, 42, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {u'app': 'kube-bench', u'controller-uid': 'bddb2b11-e845-11ea-a485-0242ac120004', u'job-name': 'kube-bench'}, 'managed_fields': None, 'name': 'kube-bench', 'namespace': 'ims-v2c26', 'owner_references': None, 'resource_version': '2586342', 'self_link': '/apis/batch/v1/namespaces/ims-v2c26/jobs/kube-bench', 'uid': 'bddb2b11-e845-11ea-a485-0242ac120004'}, 'spec': {'active_deadline_seconds': None, 'backoff_limit': 6, 'completions': 1, 'manual_selector': None, 'parallelism': 1, 'selector': {'match_expressions': None, 'match_labels': {u'controller-uid': 'bddb2b11-e845-11ea-a485-0242ac120004'}}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {u'app': 'kube-bench', u'controller-uid': 'bddb2b11-e845-11ea-a485-0242ac120004', u'job-name': 'kube-bench'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': ['kube-bench'], 'env': None, 'env_from': None, 'image': 'aquasec/kube-bench:0.3.1', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'kube-bench', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/lib/etcd', 'mount_propagation': None, 'name': 'var-lib-etcd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/lib/kubelet', 'mount_propagation': None, 'name': 'var-lib-kubelet', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/etc/systemd', 'mount_propagation': None, 'name': 'etc-systemd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/etc/kubernetes', 'mount_propagation': None, 'name': 'etc-kubernetes', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/usr/local/mount-from-host/bin', 'mount_propagation': None, 'name': 'usr-bin', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': True, 'hostname': None, 'image_pull_secrets': None, 'init_containers': None, 'node_name': None, 'node_selector': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Never', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': {'path': '/var/lib/etcd', 'type': ''}, 'iscsi': None, 'name': 'var-lib-etcd', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}, {'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': {'path': '/var/lib/kubelet', 'type': ''}, 'iscsi': None, 'name': 'var-lib-kubelet', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}, {'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': {'path': '/etc/systemd', 'type': ''}, 'iscsi': None, 'name': 'etc-systemd', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}, {'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': {'path': '/etc/kubernetes', 'type': ''}, 'iscsi': None, 'name': 'etc-kubernetes', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}, {'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': {'path': '/usr/bin', 'type': ''}, 'iscsi': None, 'name': 'usr-bin', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}, 'ttl_seconds_after_finished': None}, 'status': {'active': None, 'completion_time': None, 'conditions': None, 'failed': None, 'start_time': None, 'succeeded': None}} 2020-08-27 09:14:51,919 - functest_kubernetes.security.security - INFO - kube-bench started in 10.06 sec 2020-08-27 09:14:51,935 - kubernetes.client.rest - DEBUG - response body: {"kind":"PodList","apiVersion":"v1","metadata":{"selfLink":"/api/v1/namespaces/ims-v2c26/pods","resourceVersion":"2586380"},"items":[{"metadata":{"name":"kube-bench-59g5z","generateName":"kube-bench-","namespace":"ims-v2c26","selfLink":"/api/v1/namespaces/ims-v2c26/pods/kube-bench-59g5z","uid":"bdfe3a11-e845-11ea-a485-0242ac120004","resourceVersion":"2586379","creationTimestamp":"2020-08-27T09:14:42Z","labels":{"app":"kube-bench","controller-uid":"bddb2b11-e845-11ea-a485-0242ac120004","job-name":"kube-bench"},"ownerReferences":[{"apiVersion":"batch/v1","kind":"Job","name":"kube-bench","uid":"bddb2b11-e845-11ea-a485-0242ac120004","controller":true,"blockOwnerDeletion":true}]},"spec":{"volumes":[{"name":"var-lib-etcd","hostPath":{"path":"/var/lib/etcd","type":""}},{"name":"var-lib-kubelet","hostPath":{"path":"/var/lib/kubelet","type":""}},{"name":"etc-systemd","hostPath":{"path":"/etc/systemd","type":""}},{"name":"etc-kubernetes","hostPath":{"path":"/etc/kubernetes","type":""}},{"name":"usr-bin","hostPath":{"path":"/usr/bin","type":""}},{"name":"default-token-r2m5v","secret":{"secretName":"default-token-r2m5v","defaultMode":420}}],"containers":[{"name":"kube-bench","image":"aquasec/kube-bench:0.3.1","command":["kube-bench"],"resources":{},"volumeMounts":[{"name":"var-lib-etcd","readOnly":true,"mountPath":"/var/lib/etcd"},{"name":"var-lib-kubelet","readOnly":true,"mountPath":"/var/lib/kubelet"},{"name":"etc-systemd","readOnly":true,"mountPath":"/etc/systemd"},{"name":"etc-kubernetes","readOnly":true,"mountPath":"/etc/kubernetes"},{"name":"usr-bin","readOnly":true,"mountPath":"/usr/local/mount-from-host/bin"},{"name":"default-token-r2m5v","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Never","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"hunter-worker","hostPID":true,"securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}],"priority":0,"enableServiceLinks":true},"status":{"phase":"Succeeded","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2020-08-27T09:14:42Z","reason":"PodCompleted"},{"type":"Ready","status":"False","lastProbeTime":null,"lastTransitionTime":"2020-08-27T09:14:51Z","reason":"PodCompleted"},{"type":"ContainersReady","status":"False","lastProbeTime":null,"lastTransitionTime":"2020-08-27T09:14:51Z","reason":"PodCompleted"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2020-08-27T09:14:42Z"}],"hostIP":"172.18.0.2","podIP":"10.244.1.98","startTime":"2020-08-27T09:14:42Z","containerStatuses":[{"name":"kube-bench","state":{"terminated":{"exitCode":0,"reason":"Completed","startedAt":"2020-08-27T09:14:50Z","finishedAt":"2020-08-27T09:14:50Z","containerID":"containerd://3af84da071ca8d2e568f427fbf7971690d954aeb36956956d4c21f3bfe0f213d"}},"lastState":{},"ready":false,"restartCount":0,"image":"docker.io/aquasec/kube-bench:0.3.1","imageID":"docker.io/aquasec/kube-bench@sha256:3544f6662feb73d36fdba35b17652e2fd73aae45bd4b60e76d7ab928220b3cc6","containerID":"containerd://3af84da071ca8d2e568f427fbf7971690d954aeb36956956d4c21f3bfe0f213d"}],"qosClass":"BestEffort"}}]} 2020-08-27 09:14:52,005 - kubernetes.client.rest - DEBUG - response body: [INFO] 2 Worker Node Security Configuration [INFO] 2.1 Kubelet [PASS] 2.1.1 Ensure that the --anonymous-auth argument is set to false (Scored) [PASS] 2.1.2 Ensure that the --authorization-mode argument is not set to AlwaysAllow (Scored) [PASS] 2.1.3 Ensure that the --client-ca-file argument is set as appropriate (Scored) [FAIL] 2.1.4 Ensure that the --read-only-port argument is set to 0 (Scored) [PASS] 2.1.5 Ensure that the --streaming-connection-idle-timeout argument is not set to 0 (Scored) [FAIL] 2.1.6 Ensure that the --protect-kernel-defaults argument is set to true (Scored) [PASS] 2.1.7 Ensure that the --make-iptables-util-chains argument is set to true (Scored) [PASS] 2.1.8 Ensure that the --hostname-override argument is not set (Scored) [FAIL] 2.1.9 Ensure that the --event-qps argument is set to 0 (Scored) [FAIL] 2.1.10 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Scored) [INFO] 2.1.11 [DEPRECATED] Ensure that the --cadvisor-port argument is set to 0 [PASS] 2.1.12 Ensure that the --rotate-certificates argument is not set to false (Scored) [FAIL] 2.1.13 Ensure that the RotateKubeletServerCertificate argument is set to true (Scored) [WARN] 2.1.14 Ensure that the Kubelet only makes use of Strong Cryptographic Ciphers (Not Scored) [INFO] 2.2 Configuration Files [PASS] 2.2.1 Ensure that the kubelet.conf file permissions are set to 644 or more restrictive (Scored) [PASS] 2.2.2 Ensure that the kubelet.conf file ownership is set to root:root (Scored) [PASS] 2.2.3 Ensure that the kubelet service file permissions are set to 644 or more restrictive (Scored) [PASS] 2.2.4 Ensure that the kubelet service file ownership is set to root:root (Scored) [FAIL] 2.2.5 Ensure that the proxy kubeconfig file permissions are set to 644 or more restrictive (Scored) [FAIL] 2.2.6 Ensure that the proxy kubeconfig file ownership is set to root:root (Scored) [PASS] 2.2.7 Ensure that the certificate authorities file permissions are set to 644 or more restrictive (Scored) [PASS] 2.2.8 Ensure that the client certificate authorities file ownership is set to root:root (Scored) [PASS] 2.2.9 Ensure that the kubelet configuration file ownership is set to root:root (Scored) [PASS] 2.2.10 Ensure that the kubelet configuration file has permissions set to 644 or more restrictive (Scored) == Remediations == 2.1.4 If using a Kubelet config file, edit the file to set readOnlyPort to 0 . If using command line arguments, edit the kubelet service file /etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable. --read-only-port=0 Based on your system, restart the kubelet service. For example: systemctl daemon-reload systemctl restart kubelet.service 2.1.6 If using a Kubelet config file, edit the file to set protectKernelDefaults: true . If using command line arguments, edit the kubelet service file /etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable. --protect-kernel-defaults=true Based on your system, restart the kubelet service. For example: systemctl daemon-reload systemctl restart kubelet.service 2.1.9 If using a Kubelet config file, edit the file to set eventRecordQPS: 0 . If using command line arguments, edit the kubelet service file /etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable. --event-qps=0 Based on your system, restart the kubelet service. For example: systemctl daemon-reload systemctl restart kubelet.service 2.1.10 If using a Kubelet config file, edit the file to set tlsCertFile to the location of the certificate file to use to identify this Kubelet, and tlsPrivateKeyFile to the location of the corresponding private key file. If using command line arguments, edit the kubelet service file /etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and set the below parameters in KUBELET_CERTIFICATE_ARGS variable. --tls-cert-file= file= Based on your system, restart the kubelet service. For example: systemctl daemon-reload systemctl restart kubelet.service 2.1.13 Edit the kubelet service file /etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and set the below parameter in KUBELET_CERTIFICATE_ARGS variable. --feature-gates=RotateKubeletServerCertificate=true Based on your system, restart the kubelet service. For example: systemctl daemon-reload systemctl restart kubelet.service 2.1.14 If using a Kubelet config file, edit the file to set TLSCipherSuites: to TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256 If using executable arguments, edit the kubelet service file /etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and set the below parameter. --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256 2.2.5 Run the below command (based on the file location on your system) on the each worker node. For example, chmod 644 /etc/kubernetes/proxy.conf 2.2.6 Run the below command (based on the file location on your system) on the each worker node. For example, chown root:root /etc/kubernetes/proxy.conf == Summary == 15 checks PASS 7 checks FAIL 1 checks WARN 1 checks INFO 2020-08-27 09:14:52,006 - functest_kubernetes.security.security - WARNING - [INFO] 2 Worker Node Security Configuration [INFO] 2.1 Kubelet [PASS] 2.1.1 Ensure that the --anonymous-auth argument is set to false (Scored) [PASS] 2.1.2 Ensure that the --authorization-mode argument is not set to AlwaysAllow (Scored) [PASS] 2.1.3 Ensure that the --client-ca-file argument is set as appropriate (Scored) [FAIL] 2.1.4 Ensure that the --read-only-port argument is set to 0 (Scored) [PASS] 2.1.5 Ensure that the --streaming-connection-idle-timeout argument is not set to 0 (Scored) [FAIL] 2.1.6 Ensure that the --protect-kernel-defaults argument is set to true (Scored) [PASS] 2.1.7 Ensure that the --make-iptables-util-chains argument is set to true (Scored) [PASS] 2.1.8 Ensure that the --hostname-override argument is not set (Scored) [FAIL] 2.1.9 Ensure that the --event-qps argument is set to 0 (Scored) [FAIL] 2.1.10 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Scored) [INFO] 2.1.11 [DEPRECATED] Ensure that the --cadvisor-port argument is set to 0 [PASS] 2.1.12 Ensure that the --rotate-certificates argument is not set to false (Scored) [FAIL] 2.1.13 Ensure that the RotateKubeletServerCertificate argument is set to true (Scored) [WARN] 2.1.14 Ensure that the Kubelet only makes use of Strong Cryptographic Ciphers (Not Scored) [INFO] 2.2 Configuration Files [PASS] 2.2.1 Ensure that the kubelet.conf file permissions are set to 644 or more restrictive (Scored) [PASS] 2.2.2 Ensure that the kubelet.conf file ownership is set to root:root (Scored) [PASS] 2.2.3 Ensure that the kubelet service file permissions are set to 644 or more restrictive (Scored) [PASS] 2.2.4 Ensure that the kubelet service file ownership is set to root:root (Scored) [FAIL] 2.2.5 Ensure that the proxy kubeconfig file permissions are set to 644 or more restrictive (Scored) [FAIL] 2.2.6 Ensure that the proxy kubeconfig file ownership is set to root:root (Scored) [PASS] 2.2.7 Ensure that the certificate authorities file permissions are set to 644 or more restrictive (Scored) [PASS] 2.2.8 Ensure that the client certificate authorities file ownership is set to root:root (Scored) [PASS] 2.2.9 Ensure that the kubelet configuration file ownership is set to root:root (Scored) [PASS] 2.2.10 Ensure that the kubelet configuration file has permissions set to 644 or more restrictive (Scored) == Remediations == 2.1.4 If using a Kubelet config file, edit the file to set readOnlyPort to 0 . If using command line arguments, edit the kubelet service file /etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable. --read-only-port=0 Based on your system, restart the kubelet service. For example: systemctl daemon-reload systemctl restart kubelet.service 2.1.6 If using a Kubelet config file, edit the file to set protectKernelDefaults: true . If using command line arguments, edit the kubelet service file /etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable. --protect-kernel-defaults=true Based on your system, restart the kubelet service. For example: systemctl daemon-reload systemctl restart kubelet.service 2.1.9 If using a Kubelet config file, edit the file to set eventRecordQPS: 0 . If using command line arguments, edit the kubelet service file /etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable. --event-qps=0 Based on your system, restart the kubelet service. For example: systemctl daemon-reload systemctl restart kubelet.service 2.1.10 If using a Kubelet config file, edit the file to set tlsCertFile to the location of the certificate file to use to identify this Kubelet, and tlsPrivateKeyFile to the location of the corresponding private key file. If using command line arguments, edit the kubelet service file /etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and set the below parameters in KUBELET_CERTIFICATE_ARGS variable. --tls-cert-file= file= Based on your system, restart the kubelet service. For example: systemctl daemon-reload systemctl restart kubelet.service 2.1.13 Edit the kubelet service file /etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and set the below parameter in KUBELET_CERTIFICATE_ARGS variable. --feature-gates=RotateKubeletServerCertificate=true Based on your system, restart the kubelet service. For example: systemctl daemon-reload systemctl restart kubelet.service 2.1.14 If using a Kubelet config file, edit the file to set TLSCipherSuites: to TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256 If using executable arguments, edit the kubelet service file /etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and set the below parameter. --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256 2.2.5 Run the below command (based on the file location on your system) on the each worker node. For example, chmod 644 /etc/kubernetes/proxy.conf 2.2.6 Run the below command (based on the file location on your system) on the each worker node. For example, chown root:root /etc/kubernetes/proxy.conf == Summary == 15 checks PASS 7 checks FAIL 1 checks WARN 1 checks INFO 2020-08-27 09:14:52,008 - xtesting.ci.run_tests - INFO - Test result: +--------------------+------------------+------------------+----------------+ | TEST CASE | PROJECT | DURATION | RESULT | +--------------------+------------------+------------------+----------------+ | kube_bench | functest | 00:10 | PASS | +--------------------+------------------+------------------+----------------+ 2020-08-27 09:14:52,269 - kubernetes.client.rest - DEBUG - response body: {"kind":"Pod","apiVersion":"v1","metadata":{"name":"kube-bench-59g5z","generateName":"kube-bench-","namespace":"ims-v2c26","selfLink":"/api/v1/namespaces/ims-v2c26/pods/kube-bench-59g5z","uid":"bdfe3a11-e845-11ea-a485-0242ac120004","resourceVersion":"2586381","creationTimestamp":"2020-08-27T09:14:42Z","deletionTimestamp":"2020-08-27T09:14:52Z","deletionGracePeriodSeconds":0,"labels":{"app":"kube-bench","controller-uid":"bddb2b11-e845-11ea-a485-0242ac120004","job-name":"kube-bench"},"ownerReferences":[{"apiVersion":"batch/v1","kind":"Job","name":"kube-bench","uid":"bddb2b11-e845-11ea-a485-0242ac120004","controller":true,"blockOwnerDeletion":true}]},"spec":{"volumes":[{"name":"var-lib-etcd","hostPath":{"path":"/var/lib/etcd","type":""}},{"name":"var-lib-kubelet","hostPath":{"path":"/var/lib/kubelet","type":""}},{"name":"etc-systemd","hostPath":{"path":"/etc/systemd","type":""}},{"name":"etc-kubernetes","hostPath":{"path":"/etc/kubernetes","type":""}},{"name":"usr-bin","hostPath":{"path":"/usr/bin","type":""}},{"name":"default-token-r2m5v","secret":{"secretName":"default-token-r2m5v","defaultMode":420}}],"containers":[{"name":"kube-bench","image":"aquasec/kube-bench:0.3.1","command":["kube-bench"],"resources":{},"volumeMounts":[{"name":"var-lib-etcd","readOnly":true,"mountPath":"/var/lib/etcd"},{"name":"var-lib-kubelet","readOnly":true,"mountPath":"/var/lib/kubelet"},{"name":"etc-systemd","readOnly":true,"mountPath":"/etc/systemd"},{"name":"etc-kubernetes","readOnly":true,"mountPath":"/etc/kubernetes"},{"name":"usr-bin","readOnly":true,"mountPath":"/usr/local/mount-from-host/bin"},{"name":"default-token-r2m5v","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Never","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"hunter-worker","hostPID":true,"securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}],"priority":0,"enableServiceLinks":true},"status":{"phase":"Succeeded","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2020-08-27T09:14:42Z","reason":"PodCompleted"},{"type":"Ready","status":"False","lastProbeTime":null,"lastTransitionTime":"2020-08-27T09:14:51Z","reason":"PodCompleted"},{"type":"ContainersReady","status":"False","lastProbeTime":null,"lastTransitionTime":"2020-08-27T09:14:51Z","reason":"PodCompleted"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2020-08-27T09:14:42Z"}],"hostIP":"172.18.0.2","podIP":"10.244.1.98","startTime":"2020-08-27T09:14:42Z","containerStatuses":[{"name":"kube-bench","state":{"terminated":{"exitCode":0,"reason":"Completed","startedAt":"2020-08-27T09:14:50Z","finishedAt":"2020-08-27T09:14:50Z","containerID":"containerd://3af84da071ca8d2e568f427fbf7971690d954aeb36956956d4c21f3bfe0f213d"}},"lastState":{},"ready":false,"restartCount":0,"image":"docker.io/aquasec/kube-bench:0.3.1","imageID":"docker.io/aquasec/kube-bench@sha256:3544f6662feb73d36fdba35b17652e2fd73aae45bd4b60e76d7ab928220b3cc6","containerID":"containerd://3af84da071ca8d2e568f427fbf7971690d954aeb36956956d4c21f3bfe0f213d"}],"qosClass":"BestEffort"}} 2020-08-27 09:14:52,342 - functest_kubernetes.security.security - DEBUG - delete_namespaced_pod: {'api_version': 'v1', 'code': None, 'details': None, 'kind': 'Pod', 'message': None, 'metadata': {'_continue': None, 'resource_version': '2586381', 'self_link': '/api/v1/namespaces/ims-v2c26/pods/kube-bench-59g5z'}, 'reason': None, 'status': "{u'qosClass': u'BestEffort', u'containerStatuses': [{u'restartCount': 0, u'name': u'kube-bench', u'image': u'docker.io/aquasec/kube-bench:0.3.1', u'imageID': u'docker.io/aquasec/kube-bench@sha256:3544f6662feb73d36fdba35b17652e2fd73aae45bd4b60e76d7ab928220b3cc6', u'state': {u'terminated': {u'startedAt': u'2020-08-27T09:14:50Z', u'reason': u'Completed', u'finishedAt': u'2020-08-27T09:14:50Z', u'containerID': u'containerd://3af84da071ca8d2e568f427fbf7971690d954aeb36956956d4c21f3bfe0f213d', u'exitCode': 0}}, u'ready': False, u'lastState': {}, u'containerID': u'containerd://3af84da071ca8d2e568f427fbf7971690d954aeb36956956d4c21f3bfe0f213d'}], u'podIP': u'10.244.1.98', u'startTime': u'2020-08-27T09:14:42Z', u'hostIP': u'172.18.0.2', u'phase': u'Succeeded', u'conditions': [{u'status': u'True', u'lastProbeTime': None, u'reason': u'PodCompleted', u'type': u'Initialized', u'lastTransitionTime': u'2020-08-27T09:14:42Z'}, {u'status': u'False', u'lastProbeTime': None, u'reason': u'PodCompleted', u'type': u'Ready', u'lastTransitionTime': u'2020-08-27T09:14:51Z'}, {u'status': u'False', u'lastProbeTime': None, u'reason': u'PodCompleted', u'type': u'ContainersReady', u'lastTransitionTime': u'2020-08-27T09:14:51Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'PodScheduled', u'lastTransitionTime': u'2020-08-27T09:14:42Z'}]}"} 2020-08-27 09:14:52,462 - kubernetes.client.rest - DEBUG - response body: {"kind":"Job","apiVersion":"batch/v1","metadata":{"name":"kube-bench","namespace":"ims-v2c26","selfLink":"/apis/batch/v1/namespaces/ims-v2c26/jobs/kube-bench","uid":"bddb2b11-e845-11ea-a485-0242ac120004","resourceVersion":"2586383","creationTimestamp":"2020-08-27T09:14:42Z","deletionTimestamp":"2020-08-27T09:14:52Z","deletionGracePeriodSeconds":0,"labels":{"app":"kube-bench","controller-uid":"bddb2b11-e845-11ea-a485-0242ac120004","job-name":"kube-bench"},"finalizers":["orphan"]},"spec":{"parallelism":1,"completions":1,"backoffLimit":6,"selector":{"matchLabels":{"controller-uid":"bddb2b11-e845-11ea-a485-0242ac120004"}},"template":{"metadata":{"creationTimestamp":null,"labels":{"app":"kube-bench","controller-uid":"bddb2b11-e845-11ea-a485-0242ac120004","job-name":"kube-bench"}},"spec":{"volumes":[{"name":"var-lib-etcd","hostPath":{"path":"/var/lib/etcd","type":""}},{"name":"var-lib-kubelet","hostPath":{"path":"/var/lib/kubelet","type":""}},{"name":"etc-systemd","hostPath":{"path":"/etc/systemd","type":""}},{"name":"etc-kubernetes","hostPath":{"path":"/etc/kubernetes","type":""}},{"name":"usr-bin","hostPath":{"path":"/usr/bin","type":""}}],"containers":[{"name":"kube-bench","image":"aquasec/kube-bench:0.3.1","command":["kube-bench"],"resources":{},"volumeMounts":[{"name":"var-lib-etcd","readOnly":true,"mountPath":"/var/lib/etcd"},{"name":"var-lib-kubelet","readOnly":true,"mountPath":"/var/lib/kubelet"},{"name":"etc-systemd","readOnly":true,"mountPath":"/etc/systemd"},{"name":"etc-kubernetes","readOnly":true,"mountPath":"/etc/kubernetes"},{"name":"usr-bin","readOnly":true,"mountPath":"/usr/local/mount-from-host/bin"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Never","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","hostPID":true,"securityContext":{},"schedulerName":"default-scheduler"}}},"status":{"conditions":[{"type":"Complete","status":"True","lastProbeTime":"2020-08-27T09:14:51Z","lastTransitionTime":"2020-08-27T09:14:51Z"}],"startTime":"2020-08-27T09:14:42Z","completionTime":"2020-08-27T09:14:51Z","succeeded":1}} 2020-08-27 09:14:52,464 - functest_kubernetes.security.security - DEBUG - delete_namespaced_deployment: {'api_version': 'batch/v1', 'code': None, 'details': None, 'kind': 'Job', 'message': None, 'metadata': {'_continue': None, 'resource_version': '2586383', 'self_link': '/apis/batch/v1/namespaces/ims-v2c26/jobs/kube-bench'}, 'reason': None, 'status': "{u'completionTime': u'2020-08-27T09:14:51Z', u'conditions': [{u'status': u'True', u'lastProbeTime': u'2020-08-27T09:14:51Z', u'type': u'Complete', u'lastTransitionTime': u'2020-08-27T09:14:51Z'}], u'succeeded': 1, u'startTime': u'2020-08-27T09:14:42Z'}"} 2020-08-27 09:14:52,494 - kubernetes.client.rest - DEBUG - response body: {"kind":"Namespace","apiVersion":"v1","metadata":{"name":"ims-v2c26","generateName":"ims-","selfLink":"/api/v1/namespaces/ims-v2c26","uid":"bda52821-e845-11ea-a485-0242ac120004","resourceVersion":"2586385","creationTimestamp":"2020-08-27T09:14:42Z","deletionTimestamp":"2020-08-27T09:14:52Z"},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Terminating"}} 2020-08-27 09:14:52,496 - functest_kubernetes.security.security - DEBUG - delete_namespace: ims-v2c26 2020-08-27 09:14:54,006 - xtesting.core.testcase - DEBUG - Publishing /var/lib/xtesting/results/functest-kubernetes.log ('text/plain', None) 2020-08-27 09:14:54,449 - xtesting.core.testcase - DEBUG - Publishing /var/lib/xtesting/results/functest-kubernetes.debug.log ('text/plain', None)