API and Feature Testing requirements ==================================== Introduction ------------ The CNCF has defined a `Testing Special Interest Group `__ to make it easier for the community to write and run tests, and to contribute, analyse and act upon test results. This chapter maps the requirements written in the previous chapters as mandatory Special Interest Group Features. It enforces the overall requirements traceability to testing, especially those offered for `End-to-End Testing `__. The Anuket Reference Conformance (RC2) testing then matches the following Features tabs defined here. Kubernetes feature gate policy ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ `Feature gates `__ are a set of key-value pairs that describe Kubernetes features. The components of the control plane of Kubernetes Clusters can be run with different Feature Gate settings. A feature can be in Alpha, Beta or GA stage: - Alpha features are disabled by default, may be buggy, and support may be dropped - Beta features are enabled by default, are well tested, and support will not be dropped (although breaking API changes may happen) - GA features are stable, always enabled and cannot be disabled. The policy for RA2 to include Kubernetes features as mandatory is: Only features that are either in Beta or GA stage can be made mandatory, subject to RA2 requirements. A list of feature gates is available `here `__. Kubernetes API policy ~~~~~~~~~~~~~~~~~~~~~ The `Kubernetes API `__ supports all operations and communications between components, and external user commands. Everything in the Kubernetes platform is treated as an API object. Different API versions indicate different levels of stability and support. An API can have Alpha, Beta or Stable versions. The version of APIs that are backed by a feature will match the stage of the feature itself (i.e. Alpha, Beta or GA/Stable). The policy for RA2 to include Kubernetes APIs as mandatory is: Only APIs that are either in Beta or Stable stage can be made mandatory, subject to RA2 requirements. The Kubernetes API reference is available `here `__. The list of `API groups `__ that are mandatory is: ============================ ======================== Group Version ============================ ======================== admissionregistration.k8s.io v1 apiextensions.k8s.io v1 apiregistration.k8s.io v1 apps v1 authentication.k8s.io v1 authorization.k8s.io v1 autoscaling v1, v2, v2beta2, v2beta1 batch v1, v1beta1 certificates.k8s.io v1 coordination.k8s.io v1 core v1 discovery.k8s.io v1, v1beta1 events.k8s.io v1, v1beta1 flowcontrol.apiserver.k8s.io v1beta2, v1beta1 internal.apiserver.k8s.io v1alpha1 networking.k8s.io v1 node.k8s.io v1, v1beta1, v1alpha1 policy v1, v1beta1 rbac.authorization.k8s.io v1 scheduling.k8s.io v1 storage.k8s.io v1, v1beta1, v1alpha1 ============================ ======================== `API Machinery Special Interest Group `__ ---------------------------------------------------------------------------------------------------------------- +----------------------------------------+-------------+--------------------------------------------------------------+ | **Labels** |**Mandatory**| **Description** | +========================================+=============+==============================================================+ | Conformance | X | Kubernetes conformance test | +----------------------------------------+-------------+--------------------------------------------------------------+ | None | X | Kubernetes mainstream features | +----------------------------------------+-------------+--------------------------------------------------------------+ | Feature:ComprehensiveNamespaceDraining | X | Namespaces should always delete fast (ALL of 100 namespaces | | | | in 150 seconds) | +----------------------------------------+-------------+--------------------------------------------------------------+ | Feature:`CrossNamespacePodAffinity `__ | | | +----------------------------------------+-------------+--------------------------------------------------------------+ | Feature:`PodPriority `__ | | | +----------------------------------------+-------------+--------------------------------------------------------------+ | Feature:ScopeSelectors | X | Verify ResourceQuota with terminating scopes through scope | | | | selectors | +----------------------------------------+-------------+--------------------------------------------------------------+ | Feature:`StorageVersionAPI `__ | | | +----------------------------------------+-------------+--------------------------------------------------------------+ `Apps Special Interest Group `__ ---------------------------------------------------------------------------------------------- +----------------------------------------+-------------+--------------------------------------------------------------+ | **Labels** |**Mandatory**| **Description** | +========================================+=============+==============================================================+ | Conformance | X | Kubernetes conformance test | +----------------------------------------+-------------+--------------------------------------------------------------+ | None | X | Kubernetes mainstream features | +----------------------------------------+-------------+--------------------------------------------------------------+ | Feature:`DaemonSetUpdateSurge `__ | | | +----------------------------------------+-------------+--------------------------------------------------------------+ | Feature:`IndexedJob `__ | | | +----------------------------------------+-------------+--------------------------------------------------------------+ | Feature:`StatefulSet `__ | | | +----------------------------------------+-------------+--------------------------------------------------------------+ | Feature:StatefulUpgrade | | Stateful upgrade should maintain a functioning cluster | +----------------------------------------+-------------+--------------------------------------------------------------+ | Feature:`SuspendJob `__ | | | +----------------------------------------+-------------+--------------------------------------------------------------+ | Feature:`TaintEviction `__ | | | +----------------------------------------+-------------+--------------------------------------------------------------+ | Feature:`TTLAfterFinished `__ | | | +----------------------------------------+-------------+--------------------------------------------------------------+ `Auth Special Interest Group `__ ---------------------------------------------------------------------------------------------- +-----------------------------------------+-------------+-------------------------------------------------------------+ | **Labels** |**Mandatory**| **Description** | +=========================================+=============+=============================================================+ | Conformance | X | Kubernetes conformance test | +-----------------------------------------+-------------+-------------------------------------------------------------+ | None | X | Kubernetes mainstream features | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:`BoundServiceAccountTokenVolume | | ServiceAccount admission controller migration master | | `__ | | | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:NodeAuthenticator | X | The kubelet's main port 10250 should reject requests with | | | | no credentials | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:NodeAuthorizer | X | Setting existing and non-existent attributes should exit | | | | with the Forbidden error, not a NotFound error | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:PodSecurityPolicy | | Should enforce the restricted policy.PodSecurityPolicy | +-----------------------------------------+-------------+-------------------------------------------------------------+ | NodeFeature:FSGroup | X | ServiceAccounts should set ownership and permission when | | | | RunAsUser or FsGroup is present | +-----------------------------------------+-------------+-------------------------------------------------------------+ `Cluster Lifecycle Special Interest Group `__ ------------------------------------------------------------------------------------------------------------------------ +-----------------------------------------+-------------+-------------------------------------------------------------+ | **Labels** |**Mandatory**| **Description** | +=========================================+=============+=============================================================+ | Conformance | X | Kubernetes conformance test | +-----------------------------------------+-------------+-------------------------------------------------------------+ | None | X | Kubernetes mainstream features | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:BootstrapTokens | X | Should delete the token secret when the secret expired | +-----------------------------------------+-------------+-------------------------------------------------------------+ `Instrumentation Special Interest Group `__ -------------------------------------------------------------------------------------------------------------------- +-----------------------------------------+-------------+-------------------------------------------------------------+ | **Labels** |**Mandatory**| **Description** | +=========================================+=============+=============================================================+ | Conformance | X | Kubernetes conformance test | +-----------------------------------------+-------------+-------------------------------------------------------------+ | None | X | Kubernetes mainstream features | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:Elasticsearch | | Should check that the Kibana logging instance is alive | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature: | | Stackdriver Monitoring should have accelerator metrics | | StackdriverAcceleratorMonitoring | | | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:StackdriverCustomMetrics | | Stackdriver Monitoring should run Custom Metrics - | | | | Stackdriver Adapter for new resource model | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:StackdriverExternalMetrics | | Stackdriver Monitoring should run Custom Metrics - | | | | Stackdriver Adapter for external metrics | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:StackdriverMetadataAgent | | Stackdriver Monitoring should run Stackdriver Metadata | | | | Agent | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:StackdriverMonitoring | | | +-----------------------------------------+-------------+-------------------------------------------------------------+ `Network Special Interest Group `__ ---------------------------------------------------------------------------------------------------- +-----------------------------------------+-------------+-------------------------------------------------------------+ | **Labels** |**Mandatory**| **Description** | +=========================================+=============+=============================================================+ | Conformance | X | Kubernetes conformance test | +-----------------------------------------+-------------+-------------------------------------------------------------+ | None | X | Kubernetes mainstream features | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:Example | | Should create pod that uses DNS | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:Ingress | | Should prevent Ingress creation if more than 1 IngressClass | | | | marked as default | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:`IPv6DualStack `__ | | IPv4/IPv6 dual-stack networking is enabled by default for | | | | your Kubernetes cluster starting in 1.21, allowing the | | | | simultaneous assignment of both IPv4 and IPv6 addresses. | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:kubemci | | Should create ingress with pre-shared certificate | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:KubeProxyDaemonSetMigration | | Upgrade kube-proxy from static pods to a DaemonSet should | | | | maintain a functioning cluster | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:KubeProxyDaemonSetUpgrade | | Upgrade kube-proxy from static pods to a DaemonSet should | | | | maintain a functioning cluster | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:NEG | | Should sync endpoints to NEG | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:NoSNAT | X | Should be able to send traffic between Pods without SNAT | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:Networking-IPv4 | X | Networking should provide Internet connection for | | | | containers | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:Networking-IPv6 | | Networking should provide Internet connection for | | | | containers | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:Networking-Performance | X | run iperf2 | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:NetworkPolicy | | NetworkPolicy between server and client should enforce | | | | policy to allow traffic only from a different namespace, | | | | based on NamespaceSelector | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:PerformanceDNS | | Should answer DNS query for maximum number of services per | | | | cluster | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:SCTP | | should allow creating a basic SCTP service with pod and | | | | endpoints | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:SCTPConnectivity | | Pods should function for intra-pod communication: sctp | +-----------------------------------------+-------------+-------------------------------------------------------------+ `Node Special Interest Group `__ ---------------------------------------------------------------------------------------------- +-----------------------------------------+-------------+-------------------------------------------------------------+ | **Labels** |**Mandatory**| **Description** | +=========================================+=============+=============================================================+ | Conformance | X | Kubernetes conformance test | +-----------------------------------------+-------------+-------------------------------------------------------------+ | None | X | Kubernetes mainstream features | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:Example | X | Liveness pods should be automatically restarted | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature: | | Resource tracking for 100 pods per node | | ExperimentalResourceUsageTracking | | | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:GPUUpgrade | | Master upgrade should NOT disrupt GPU Pod | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:PodGarbageCollector | | Should handle the creation of 1000 pods | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:RegularResourceUsageTracking | | Resource tracking for 0 pods per node | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:`ProbeTerminationGracePeriod `__ | | | +-----------------------------------------+-------------+-------------------------------------------------------------+ | NodeFeature:`DownwardAPIHugePages `__ | | | +-----------------------------------------+-------------+-------------------------------------------------------------+ | NodeFeature:`PodReadinessGate `__ | | | +-----------------------------------------+-------------+-------------------------------------------------------------+ | NodeFeature:RuntimeHandler | | RuntimeClass should run a Pod requesting a RuntimeClass | | | | with a configured handler | +-----------------------------------------+-------------+-------------------------------------------------------------+ | NodeFeature:`Sysctls `__ | | | +-----------------------------------------+-------------+-------------------------------------------------------------+ `Scheduling Special Interest Group `__ ---------------------------------------------------------------------------------------------------------- +-----------------------------------------+-------------+-------------------------------------------------------------+ | **Labels** |**Mandatory**| **Description** | +=========================================+=============+=============================================================+ | Conformance | X | Kubernetes conformance test | +-----------------------------------------+-------------+-------------------------------------------------------------+ | None | X | Kubernetes mainstream features | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:GPUDevicePlugin | | Run Nvidia GPU Device Plugin tests | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:`LocalStorageCapacityIsolation | X | Validates local ephemeral storage resource limits of pods | | `__ | | | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:Recreate | | Run Nvidia GPU Device Plugin tests with a recreation | +-----------------------------------------+-------------+-------------------------------------------------------------+ `Storage Special Interest Group `__ ---------------------------------------------------------------------------------------------------- +-----------------------------------------+-------------+-------------------------------------------------------------+ | **Labels** |**Mandatory**| **Description** | +=========================================+=============+=============================================================+ | Conformance | X | Kubernetes conformance test | +-----------------------------------------+-------------+-------------------------------------------------------------+ | None | X | Kubernetes mainstream features | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:ExpandInUsePersistentVolumes | | | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:Flexvolumes | | | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:GKELocalSSD | | | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:VolumeSnapshotDataSource | | | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:Volumes | X | | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:vsphere | | | +-----------------------------------------+-------------+-------------------------------------------------------------+ | Feature:Windows | | | +-----------------------------------------+-------------+-------------------------------------------------------------+ | NodeFeature:EphemeralStorage | X | | +-----------------------------------------+-------------+-------------------------------------------------------------+ | NodeFeature:FSGroup | X | | +-----------------------------------------+-------------+-------------------------------------------------------------+