API and Feature Testing requirements ==================================== Introduction ------------ The CNCF has defined a `Testing Special Interest Group `__ to make it easier for the community to write and run tests, and to contribute, analyse and act upon test results. This chapter maps the requirements written in the previous chapters as mandatory Special Interest Group Features. It enforces the overall requirements traceability to testing, especially those offered for `End-to-End Testing `__. The Anuket Reference Conformance (RC2) testing then matches the following Features tabs defined here. Kubernetes feature gate policy ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ `Feature gates `__ are a set of key-value pairs that describe Kubernetes features. The components of the control plane of Kubernetes Clusters can be run with different Feature Gate settings. A feature can be in Alpha, Beta or GA stage: - Alpha features are disabled by default, may be buggy, and support may be dropped - Beta features are enabled by default, are well tested, and support will not be dropped (although breaking API changes may happen) - GA features are stable, always enabled and cannot be disabled. The policy for RA2 to include Kubernetes features as mandatory is: Only features that are either in Beta or GA stage can be made mandatory, subject to RA2 requirements. A list of feature gates is available `here `__. Kubernetes API policy ~~~~~~~~~~~~~~~~~~~~~ The `Kubernetes API `__ supports all operations and communications between components, and external user commands. Everything in the Kubernetes platform is treated as an API object. Different API versions indicate different levels of stability and support. An API can have Alpha, Beta or Stable versions. The version of APIs that are backed by a feature will match the stage of the feature itself (i.e. Alpha, Beta or GA/Stable). The policy for RA2 to include Kubernetes APIs as mandatory is: Only APIs that are either in Beta or Stable stage can be made mandatory, subject to RA2 requirements. The Kubernetes API reference is available `here `__. The list of `API groups `__ that are mandatory is: ============================ ==================== Group Version ============================ ==================== admissionregistration.k8s.io v1 apiextensions.k8s.io v1 apiregistration.k8s.io v1 apps v1 authentication.k8s.io v1 authorization.k8s.io v1 autoscaling v1, v2beta2, v2beta1 batch v1 certificates.k8s.io v1 coordination.k8s.io v1 core v1 discovery.k8s.io v1 events.k8s.io v1 flowcontrol.apiserver.k8s.io v1beta1 networking.k8s.io v1 node.k8s.io v1 policy v1 rbac.authorization.k8s.io v1 scheduling.k8s.io v1 storage.k8s.io v1 ============================ ==================== `API Machinery Special Interest Group `__ ---------------------------------------------------------------------------------------------------------------- ====================================================================================================================================================== ============= ========================================================================================= **Labels** **Mandatory** **Description** ====================================================================================================================================================== ============= ========================================================================================= Conformance X Kubernetes conformance test None X Kubernetes mainstream features Feature:ComprehensiveNamespaceDraining X Namespaces should always delete fast (ALL of 100 namespaces in 150 seconds) Feature:`CrossNamespacePodAffinity `__ Should verify ResourceQuota with cross namespace pod affinity scope using scope-selectors Feature:`PodPriority `__ X Verify ResourceQuota's priority class scope against a pod with different priority class Feature:ScopeSelectors X Verify ResourceQuota with terminating scopes through scope selectors Feature:`StorageVersionAPI `__ ====================================================================================================================================================== ============= ========================================================================================= `Apps Special Interest Group `__ ---------------------------------------------------------------------------------------------- ====================================================================================================================================== ============= ====================================================================================================================================================== **Labels** **Mandatory** **Description** ====================================================================================================================================== ============= ====================================================================================================================================================== Conformance X Kubernetes conformance test None X Kubernetes mainstream features Feature:`DaemonSetUpdateSurge `__ Daemon set should surge pods onto nodes when spec was updated and update strategy is RollingUpdate Feature:`IndexedJob `__ Should create pods for an Indexed job with completion indexes Feature:`StatefulSet `__ Should creating a working zookeeper cluster Feature:StatefulUpgrade Stateful upgrade should maintain a functioning cluster Feature:`SuspendJob `__ Should not create pods when created in suspend state Feature:`TaintEviction `__ All pods on the unreachable node should be marked as NotReady upon the node turn NotReady AND all pods should be evicted after eviction timeout passes Feature:`TTLAfterFinished `__ X Job should be deleted once it finishes after TTL seconds ====================================================================================================================================== ============= ====================================================================================================================================================== `Auth Special Interest Group `__ ---------------------------------------------------------------------------------------------- ============================================================================================================================================================= ============= ======================================================================================================= **Labels** **Mandatory** **Description** ============================================================================================================================================================= ============= ======================================================================================================= Conformance X Kubernetes conformance test None X Kubernetes mainstream features Feature:`BoundServiceAccountTokenVolume `__ ServiceAccount admission controller migration master upgrade should maintain a functioning cluster Feature:NodeAuthenticator X The kubelet's main port 10250 should reject requests with no credentials Feature:NodeAuthorizer X Setting existing and non-existent attributes should exit with the Forbidden error, not a NotFound error Feature:PodSecurityPolicy Should enforce the restricted policy.PodSecurityPolicy NodeFeature:FSGroup X ServiceAccounts should set ownership and permission when RunAsUser or FsGroup is present ============================================================================================================================================================= ============= ======================================================================================================= `Cluster Lifecycle Special Interest Group `__ ------------------------------------------------------------------------------------------------------------------------ ======================= ============= ====================================================== **Labels** **Mandatory** **Description** ======================= ============= ====================================================== Conformance X Kubernetes conformance test None X Kubernetes mainstream features Feature:BootstrapTokens X Should delete the token secret when the secret expired ======================= ============= ====================================================== `Instrumentation Special Interest Group `__ -------------------------------------------------------------------------------------------------------------------- ======================================== ============= ============================================================================================= **Labels** **Mandatory** **Description** ======================================== ============= ============================================================================================= Conformance X Kubernetes conformance test None X Kubernetes mainstream features Feature:Elasticsearch Should check that the Kibana logging instance is alive Feature:StackdriverAcceleratorMonitoring Stackdriver Monitoring should have accelerator metrics Feature:StackdriverCustomMetrics Stackdriver Monitoring should run Custom Metrics - Stackdriver Adapter for new resource model Feature:StackdriverExternalMetrics Stackdriver Monitoring should run Custom Metrics - Stackdriver Adapter for external metrics Feature:StackdriverMetadataAgent Stackdriver Monitoring should run Stackdriver Metadata Agent Feature:StackdriverMonitoring ======================================== ============= ============================================================================================= `Network Special Interest Group `__ ---------------------------------------------------------------------------------------------------- =============================================================================================== ============= ====================================================================================================================================================================================================================================================================================== **Labels** **Mandatory** **Description** =============================================================================================== ============= ====================================================================================================================================================================================================================================================================================== Conformance X Kubernetes conformance test None X Kubernetes mainstream features Feature:Example Should create pod that uses DNS Feature:Ingress Should prevent Ingress creation if more than 1 IngressClass marked as default Feature:`IPv6DualStack `__ IPv4/IPv6 dual-stack networking enables the allocation of both IPv4 and IPv6 addresses to Pods and Services. IPv4/IPv6 dual-stack networking is enabled by default for your Kubernetes cluster starting in 1.21, allowing the simultaneous assignment of both IPv4 and IPv6 addresses. Feature:kubemci Should create ingress with pre-shared certificate Feature:KubeProxyDaemonSetMigration Upgrade kube-proxy from static pods to a DaemonSet should maintain a functioning cluster Feature:KubeProxyDaemonSetUpgrade Upgrade kube-proxy from static pods to a DaemonSet should maintain a functioning cluster Feature:NEG Should sync endpoints to NEG Feature:NoSNAT X Should be able to send traffic between Pods without SNAT Feature:Networking-IPv4 X Networking should provide Internet connection for containers Feature:Networking-IPv6 Networking should provide Internet connection for containers Feature:Networking-Performance X run iperf2 Feature:NetworkPolicy NetworkPolicy between server and client should enforce policy to allow traffic only from a different namespace, based on NamespaceSelector Feature:PerformanceDNS Should answer DNS query for maximum number of services per cluster Feature:SCTP should allow creating a basic SCTP service with pod and endpoints Feature:SCTPConnectivity Pods should function for intra-pod communication: sctp =============================================================================================== ============= ====================================================================================================================================================================================================================================================================================== `Node Special Interest Group `__ ---------------------------------------------------------------------------------------------- ========================================================================================================================================================================================= ============= ==================================================================================================================================== **Labels** **Mandatory** **Description** ========================================================================================================================================================================================= ============= ==================================================================================================================================== Conformance X Kubernetes conformance test None X Kubernetes mainstream features Feature:Example X Liveness pods should be automatically restarted Feature:ExperimentalResourceUsageTracking Resource tracking for 100 pods per node Feature:GPUUpgrade Master upgrade should NOT disrupt GPU Pod Feature:PodGarbageCollector Should handle the creation of 1000 pods Feature:RegularResourceUsageTracking Resource tracking for 0 pods per node Feature:`ProbeTerminationGracePeriod `__ X Probing container should override timeoutGracePeriodSeconds when LivenessProbe field is set NodeFeature:`DownwardAPIHugePages `__ Downward API tests for huge pages should provide container's limits.hugepages-pagesize; and requests.hugepages-pagesize& as env vars NodeFeature:`PodReadinessGate `__  X Pods should support pod readiness gates NodeFeature:RuntimeHandler RuntimeClass should run a Pod requesting a RuntimeClass with a configured handler NodeFeature:`Sysctls `__ X Should not launch unsafe, but not explicitly enabled sysctls on the node ========================================================================================================================================================================================= ============= ==================================================================================================================================== `Scheduling Special Interest Group `__ ---------------------------------------------------------------------------------------------------------- ========================================================================================================================== ============= ================================================================================= **Labels** **Mandatory** **Description** ========================================================================================================================== ============= ================================================================================= Conformance X Kubernetes conformance test None X Kubernetes mainstream features Feature:GPUDevicePlugin Run Nvidia GPU Device Plugin tests Feature:`LocalStorageCapacityIsolation `__ X Validates local ephemeral storage resource limits of pods that are allowed to run Feature:Recreate Run Nvidia GPU Device Plugin tests with a recreation ========================================================================================================================== ============= ================================================================================= `Storage Special Interest Group `__ ---------------------------------------------------------------------------------------------------- ==================================== ============= ============================== **Labels** **Mandatory** **Description** ==================================== ============= ============================== Conformance X Kubernetes conformance test None X Kubernetes mainstream features Feature:ExpandInUsePersistentVolumes Feature:Flexvolumes Feature:GKELocalSSD Feature:VolumeSnapshotDataSource Feature:Volumes X Feature:vsphere Feature:Windows NodeFeature:EphemeralStorage X NodeFeature:FSGroup X ==================================== ============= ==============================