3. Cloud Infrastructure + VNF Target State & Specification¶
3.1. Introduction¶
The VNF profile is used to describe every workload running on top of Cloud Infrastructure. The Cloud Infrastructure SW profile is used to describe the list of features provided by the hypervisor and host OS.
The CNTT Reference Model will be referenced as RM to avoid long and duplicated reference titles.
3.2. VNF profile¶
Any virtual network functions and/or cloud-native network functions must choose to run on one of the pre-defined of entries in Cloud Infrastructure Infrastructure Profiles Catalogue. As states in Profiles and Workload Flavours BMC_PRE_CONFIGURED, the entry uses the following naming convention.
B/N <I opt> . <Flavour> . <S ext> . <A ext>
B/N is used to specify the instance type (Basic, Network Intensive), different instance types are associated with different acceleration extensions, network characteristics (RM: Profiles Specifications & Capability Mapping :ref:`ref_model:chapters/chapter04:profiles-specifications-capability-mapping).
Whereas:
<I opt>stands for network interface options, e.g., the range of vNIC Bandwidth of B instance shall be selected from n1 to n60, for C instance is from n10 to n300, for N instance is from n10 to n600. (RM: 4.2.2 Virtual Network Interface Specifications and Table 4-23: Virtual NIC Interfaces Options)Instance capabilities do not explicitly appear on the naming convention, because some are common to all instance types so they are covered in the
<Flavour>(RM: 4.2.1 Compute Flavours), additionally there are a few capabilities which bind to certain types, e.g., CPU pinning or NUMA support are only available for N instances, while CPU overbooking can only happen to B instance types but not for N. (See RM: Table 4-24 Mapping of Cloud Infrastructure Capabilities to Instance Types for full mapping details)<S ext>stands for persistent storage extensions, contains the size and the performance settings (RM: Table 4-20: Storage Extension Options), note the storage extension is common to all instance types as<A ext>stands for accelaration extensions, features like Transcoding and Programmable are associated with C instances (RM: 4.2.4.3.1 Compute Accleration Extensions), while IPSec and Crypto features only make scene with N instance (RM: 4.2.4.2.1 Network Acceleration Extensions),
Every VNF instance must declare its profiles explicitly, which can be used by VIM to allocate resources duration instantiation, also it would be useful to evaluate portability of workload.
.conf |
Basic |
Network Intensive |
|---|---|---|
#vCPU cores |
Per selected <Flavour> |
Per selected <Flavour> |
Amount of RAM |
Per selected <Flavour> |
Per selected <Flavour> |
Total instance (ephemeral) storage (GB) |
Per selected <Flavour> |
Per selected <Flavour> |
Total instance (persistent) storage (GB) |
Per selected |
Per selected |
n1, n2, n3, n4, n5, n6 |
Y |
N |
n10, n20, n30, n40, n50, n60 |
Y |
Y |
n25, n50, n75, n100, n125, n150 |
N |
Y |
n50, n100, n150, n200, n250, n300 |
N |
Y |
n100, n200, n300, n400, n500, n600 |
N |
Y |
CPU pinning support |
N |
Y |
NUMA support |
N |
Y |
IPSec Acceleration) |
N |
Y |
Crypto Acceleration |
N |
Y |
Transcoding Acceleration |
N |
N |
Programmable Acceleration |
N |
N |
Enhanced Cache Management |
E |
E |
Monitoring of L2-7 data |
N |
Y |
CPU overbooking |
1:4 |
1:1 |
vNIC QoS |
N |
Y |
Huge page support |
N |
Y |
Host CPU usage |
Y |
Y |
Virtual compute resource (vCPU) usage |
Y |
Y |
Host CPU utilization |
Y |
Y |
Virtual compute resource (vCPU) utilization |
Y |
Y |
External storage capacity |
N |
N |
Open Point 1: Does ONAP have some relevant spec or VNF declaration schema so that CNTT can re-use/revise to cover what we need ? Or define a new one ?
Open Point 2: What principles should be followed if some the pre-define VNF profile items does not match what actual requires ? How to adjust, “ceiling”, “floor”, “customise” ?
3.3. Cloud Infrastructure SW profile¶
Cloud Infrastructure Software Profiles features and requirements defines the Cloud Infrastructure software layer. The profile depicts the feature status of the
virtual Compute (nfvi.com.cfg.xxx in RM Table 5-7: Virtual Compute features and configuration for the 3 types of SW profiles and nfvi.com.acc.cfg.xxx in Table 5-8: Virtual Compute Acceleration features),
storage (nfvi.stg.cfg.xxx in RM1: Table 5-9: Virtual Storage features and configuration for the 3 types of SW profiles and nfvi.stg.acc.cfg.xxx in Table 5-10: Virtual Storage Acceleration features)
networking configuration(see nfvi.net.cfg.xxx in Table 5-11 Virtual Networking features and configuration for the 3 types of SW profiles and nfvi.net.acc.cfg.xxx in Table 5-12 Virtual Networking Acceleration features)
This profile is the global settings for the whole Cloud Infrastructure, which means there should be only one entry per Cloud Infrastructure resource pool, i.e., Basic/Network
.conf |
Basic |
Network Intensive |
|---|---|---|
CPU allocation ratio |
4:1 |
1:1 |
NUMA awareness |
N |
Y |
CPU pinning capability |
N |
Y |
Huge pages |
N |
Y |
Catalogue storage Types |
Y |
Y |
Storage Block |
Y |
Y |
Storage Object |
Y |
Y |
Storage with replication |
N |
Y |
Storage with encryption |
Y |
Y |
Storage IOPS oriented |
N |
Y |
Storage capacity oriented |
N |
N |
vNIC interface |
virtio1.1 |
virtio1.1 |
Overlay protocol |
VXLAN, MPLSoUDP, GENEVE, other |
VXLAN, MPLSoUDP, GENEVE, other |
NAT |
Y |
Y |
Security Group |
Y |
Y |
SFC support |
N |
Y |
Traffic patterns symmetry |
Y |
Y |
vSwitch optimisation |
N |
Y, DPDK |
Support of HW offload |
N |
Y, support of SmartNic |
Crypto acceleration |
N |
Y |
Crypto Acceleration Interface |
N |
Y |
3.4. Cloud Infrastructure Hardware Profile¶
Cloud Infrastructure Hardware Profiles features and requirements. defines the Cloud Infrastructure hardware layer profiles.The labs are typically provisioned with the minimal required hardware and thus it is difficult to partition the available hardware to provision/configure multiple Cloud Infrastructure profiles. However, when reference implementations and the follow up testing and verification are conducted, the hardware profile need to be clearly described. This is especially important for performance testing and verification.
Reference |
Feature |
Description |
Basic Type |
Network Intensive |
|---|---|---|---|---|
nfvi.hw.cpu.cfg.001 |
Number of CPU sockets |
This determines the minimum number of CPU sockets within each host |
2 |
2 |
nfvi.hw.cpu.cfg.002 |
Number of cores per CPU |
This determines the number of cores needed per each CPU. |
20 |
20 |
nfvi.hw.cpu.cfg.003 |
NUMA |
NUMA support and BIOS configured to enable NUMA |
N |
Y |
nfvi.hw.cpu.cfg.004 |
Simultaneous Multithreading (SMT) |
This permits multiple independent threads of execution on a single core. |
Y |
Y |
nfvi.hw.cac.cfg.001 |
GPU |
GPU |
N |
N |
nfvi.hw.stg.hdd.cfg.001* |
Local Storage HDD |
Hard Disk Drive |
||
nfvi.hw.stg.ssd.cfg.002* |
Local Storage SSD |
Solid State Drive |
Recommended |
Recommended |
nfvi.hw.nic.cfg.001 |
NIC Ports |
Total Number of NIC Ports available in the host |
4 |
4 |
nfvi.hw.nic.cfg.002 |
Port Speed |
Port speed specified in Gbps (minimum values) |
10 |
25 |
nfvi.hw.pci.cfg.001 |
PCIe slots |
Number of PCIe slots available in the host |
8 |
8 |
nfvi.hw.pci.cfg.002 |
PCIe speed |
Gen 3 |
Gen 3 |
|
nfvi.hw.pci.cfg.003 |
PCIe Lanes |
8 |
8 |
|
nfvi.hw.bdc.cfg.001 |
Bonded VLAN ports |
Y |
Y |
|
nfvi.hw.nac.cfg.001 |
Cryptographic Acceleration |
IPSec, Crypto |
N |
Optional |
nfvi.hw.nac.cfg.002 |
SmartNIC |
A SmartNIC that is used to offload network functionality to hardware |
N |
Optional |
nfvi.hw.nac.cfg.003 |
Compression |
3.5. Cloud Infrastructure Required State¶
This sections describes the readiness of Cloud Infrastructure before the certification process can begin. Once the Cloud Infrastructure is configured with either of the profiles - B, N, a set of tests (for example functests) should be run in order to determine the readiness of the Cloud Infrastructure for certification. #TODO : Identify the tests for this section
Architecture and OpenStack Requirements describes the requirements related to the following 8 domains: general(gen), infrastructure(inf), VIM(vim), Interface & API(int), Tenants(tnt), LCM(lcm), Assurance(asr), Security(sec).
Ref # |
Description |
|---|---|
|
must use OpenStack APIs. |
|
must support dynamic request and configuration of virtual resources through APIs. |
|
should consist of stateless service components. However, where state is required it must be kept external to the components. |
|
should consist of service components implemented as microservices that are individually dynamically scalable. |
|
should support policy driven auto-scaling. |
|
must support resilient OpenStack components that are required for the continued availability of running workloads. |
|
should support resilient OpenStack service components that are not subject to |
|
must provide High Availability for OpenStack components. |
|
must provide compute resources for VM instances. |
|
should include industry standard hardware management systems at both HW device and platform level |
|
should support symmetrical CPU multi-processing with shared memory access as well as multi-threading. |
|
must be able to support multiple CPU Types to support Base, Network Intensive infrastructure profiles. |
|
must support Hardware Platforms with NUMA capabilities. |
|
must support CPU Pinning. |
|
must support different hardware configurations to support Base, Network Intensive infrastructure profiles. |
|
must provide shared Block storage for VM Instances. |
|
must provide shared Object storage for VM Instances. |
|
may provide local file system storage solution for VM Instances. |
|
may support Software Defined Storage (SDS) that seamlessly supports shared block storage, object storage and flat files. |
|
should be able to accommodate VNFs that store back into its image through use of hypervisor attached volumes. |
|
should make the immutable images available via location independent means. |
|
should provide high-performance and horizontally scalable VM storage. |
|
should allow use of externally provided large archival storage for its Backup / Restore / Archival needs. |
|
should make available all non-host OS / Hypervisor / Host systems storage as network-based Block, File or Object Storage for tenant/management consumption. |
|
must provide virtual network interfaces to VM instances. |
|
must include capabilities for integrating SDN controllers to support provisioning of network services, from the OpenStack Neutron service, such as networking of VTEPs to the Border Edge based VRFs. |
|
must support low latency and high throughput traffic needs. |
|
should support service function chaining. |
|
must allow for East/West tenant traffic within the cloud (via tunnelled encapsulation overlay such as VXLAN or Geneve). |
|
should support Distributed Virtual Routing (DVR) to allow compute nodes to route traffic efficiently. |
|
must support network resiliency. |
|
The NFVI Network Fabric should embrace the concepts of open networking and disaggregation using commodity networking hardware and disaggregated Network Operating Systems. |
|
The NFVI Network Fabric should embrace open-based standards and technologies. |
|
The NFVI Network Fabric must be capable of supporting highly available (Five 9’s or better) VNF workloads. |
|
The NFVI Network Fabric should be architected to provide a standardised, scalable, and repeatable deployment model across all applicable NFVI sites. |
|
The SDN solution should be configurable via orchestration or VIM systems in an automated manner using openly published API definitions. |
|
The SDN solution should be able to support federated networks. |
|
The SDN solution should be able to be centrally administrated and configured. |
|
must support multiple networking options for NFVI to support Base, Network Intensive infrastructure profiles. |
|
must support dual stack IPv4 and IPv6 for tenant networks and workloads. |
|
should use dual stack IPv4 and IPv6 for NFVI internal networks. |
|
should support Application Specific Acceleration (exposed to VNFs). |
|
should support NFVI Acceleration (such as SmartNICs). |
|
should not rely on SR-IOV PCI-Pass through to provide acceleration to VNFs. |
|
must allow infrastructure resource sharing. |
|
should support deployment of OpenStack components in containers. |
|
must allow VIM to discover and manage NFVI resources. |
|
must support Enhanced Platform Awareness (EPA). |
|
must include image repository management. |
|
must allow orchestration solutions to be integrated with VIM. |
|
must support a multi-tenanted environment. |
|
must support resource tagging. |
|
must support horizontal scaling. |
|
must provide Control API endpoints to cloud platform core services. |
|
must provide GUI access to tenant facing cloud platform core services. |
|
must provide APIs needed to discover and manage NFVI resources. |
|
should provide an open and standard acceleration interface to VNFs. |
|
should not rely on SR-IOV PCI-Pass through for acceleration interface exposed to VNFs. |
|
must support multi-tenancy. |
|
must support self-service dashboard (GUI) and APIs for users to deploy, configure and manage their workloads. |
|
must support zero downtime expansion/change of physical capacity (compute hosts, storage increase/replacement). |
|
should allow for “cookie cutter” automated deployment, configuration, provisioning and management of multiple NFVI sites. |
|
must support hitless upgrades of software provided by the cloud provider so that the availability of running workloads is not impacted. |
|
should support hitless upgrade of all software provided by the cloud provider that are not covered by |
|
should support declarative specifications of hardware and software assets for automated deployment, configuration, maintenance and management. |
|
should support automated process for Deployment and life-cycle management of VIM Instances. |
|
should support integrating with CI/CD Toolchain for NFVI and VIM components Automation. |
|
must include integration with various infrastructure components to support collection of telemetry for assurance monitoring and network intelligence. |
|
should support Network Intelligence capabilities that allow richer diagnostic capabilities which take as input broader set of data across the network and from VNF workloads. |
|
must allow for the collection and dissemination of performance and fault information. |
|
The NFVI Network Fabric and Network Operating System must provide network operational visibility through alarming and streaming telemetry services for operational management, engineering planning, troubleshooting, and network performance optimisation. |
|
must provide tenant isolation. |
|
must support policy based RBAC. |
|
must support a centralised authentication and authorisation mechanism. |
|
must support identity management (specific roles and permissions assigned to a domain or tenant). |
|
must support password encryption. |
|
must support data, at-rest and in-flight, encryption. |
|
must support integration with Corporate Identity Management systems. |
|
must comply with all applicable standards and regulations. |
|
must comply with all applicable regional standards and regulations. |
|
must have the underlay network include strong access controls that comply with ISO 27001 and adhere to the V1.1 NIST Cybersecurity Framework. |
|
must have all security logs stored in accordance with ISO27001. |
|
must have the underlay network incorporate encrypted and/or private communications channels to ensure its security. |
|
must configure all of the underlay network components to ensure the complete separation from the overlay customer deployments. |
RA1: Chapter 5 Interfaces and APIs describes the baseline version regarding to OpenStack Service APIs.
OpenStack Service |
Link for API list |
Baseline Version |
Minimal API Microversion |
|---|---|---|---|
Identity: Keystone |
https://docs.openstack.org/api-ref/identity/v3/index.html?expanded=#identity-api-operations |
3 |
3.8 |
Compute: Nova |
v2.1 |
2.53 |
|
Networking: Neutron |
v2.0 |
NA |
|
Imaging: Glance |
https://docs.openstack.org/api-ref/image/v2/index.html#images |
v2 |
2.5 |
Block Storage: Cinder |
https://docs.openstack.org/api-ref/block-storage/v3/index.html#api-versions |
v3 |
3.43 |
Object Storage: Swift |
v1 |
NA |
|
Orchestration: Heat |
https://docs.openstack.org/api-ref/orchestration/v1/index.html#api-versions |
v1.0 |
NA |
Acceleration: Cyborg |
v1.0 |
NA |
3.6. Cloud Infrastructure and VIM Architecture¶
This sections concludes the expectation for Cloud Infrastructure and VIM architecture according to RA1: Chapter 3 Cloud Infrastructure + VIM Architecture
Requirement Area |
Description |
|---|---|
Multi-Tenancy |
permit to host several VNF projects with the insurance to have isolated environment for each. Naming and quotas are kept consistent (details TBD) |
Virtual Compute |
The virtual compute resources (vCPU and vRAM) used by the VNFs behave like their physical counterparts. The configuration of the virtual resources will depend on the profile and the flavour needed to host VNF components. |
Virtual Storage |
The three storage services offered by NFVI are:Persistent storage, Ephemeral storage, Image storage |
Virtual Networking Neutron standalone |
Allows users to create networks, subnets, ports, routers etc. Facilitates traffic isolation between different subnets. Support multiple network segments. Create routers to connect layer 2 networks |
Virtual Networking – 3rd party SDN solution |
Utilize OpenStack Neutron to support plugins for various SDN controllers include the standard ML-2 plugin and vendor product specific monolithic plugins. |
Acceleration |
The hardware accelerator covers the options for ASICs, SmartNIC, FPGAs, GPU etc. to offload the main CPU, and to accelerate workload performance. NFVI should manage the accelerators by plugins and provide the acceleration capabilities to VNFs.With the acceleration abstraction layer defined, hardware accelerators as well as software accelerators can be abstracted as a set of acceleration functions (or acceleration capabilities) which exposes a common API to either the VNF or the host. |
VIM core services |
horizon, heat, keystone, nova, neutron, cinder, glance, swift, Ironic(optional only for bare-metal management) |
Foundation services |
Foundation Node To build and lifecycle manage an OpenStack cloud it is typically necessary to deploy a server or virtual machine as a deployment node. This function must be able to manage the bare-metal provisioning of the hardware resources( can be detached from the OpenStack cloud). Capabilities include building the cloud (control, compute, storage, network hardware resources), Patch management / upgrades / change management, Grow / Shrink resources |
Cloud Controller Services |
All components must be deployed within a high available architecture that can withstand at least a single node failure and respects the anti-affinity rules for the location of the services |
Physical Network |
The recommended network architecture is spine and leaf topology; however, for small sites, a legacy topology (access/aggregation switches) can be set up. |
3.7. Cloud Infrastructure and VIM Component Level Architecture¶
This sections concludes the expectation for Cloud Infrastructure and VIM component level architecture according to RA1: Chapter 4 Cloud Infrastructure + VIM Component Level Architecture
Requirement for control node:
Requirement Area |
Description |
|---|---|
SLA |
Minimum 3 nodes for high availability |
HW specifications |
Boot disks are dedicated with Flash technology disks |
Requirement for compute node:
Requirement Area |
Description |
|---|---|
BIOS requirement |
boot parameters should follow the table defined in Compute Nodes |
SLA |
minimum: two nodes per profile |
sizing rules |
should follow the table defined in Compute Nodes |
Requirement for network fabric:
Requirement Area |
Description |
|---|---|
Network Layout |
should follow the table in High Level Logical Network Layout |
Consumable Infrastructure Resources and Services
Requirement Area |
Description |
|---|---|
Support for Profiles and T-shirt instance types |
should follow tabels specified in Support for Cloud Infrastructure Profiles and flavors |
Availability |
The NFVI doesn’t provide any resiliency mechanisms at the service level. Any VM restart shall be triggered by the VNF Manager instead of OpenStack |
NUMA |
For Network intensive instances, VNF Component should fit into a single NUMA zone for performance reason |
3.8. Interface and API for Reference Implementation 1¶
The following table lists the interface for RI1.
OpenStack Service |
Link for API list |
API Version |
Minimal API Microversion |
|---|---|---|---|
Identity: Keystone |
3 |
3.8 |
|
Compute: Nova |
v2.1 |
2.53 |
|
Networking: Neutron |
v2.0 |
||
Image: Glance |
v2 |
2.5 |
|
Block Storage: Cinder |
v3 |
3.43 |
|
Object Storage: Swift |
v1 |
||
Placement |
v1 |
1.10 |
|
Orchestration: Heat |
v1 |
||
Acceleration: Cyborg |
v2 |
||
K8S API |
https://kubernetes.io/docs/concepts/overview/kubernetes-api/ |
||
KVM APIs |
https://www.kernel.org/doc/Documentation/virtual/kvm/api.txt |
||
Libvirt APIs |