2. Architecture Requirements

2.1. Introduction

This chapter will use the requirements defined in the overall Reference Model and only make additional entries in section 2.3 if there are additional requirements needed for this Reference Architecture.

2.2. Definitions

The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in RFC2119.

2.3. Reference Model Requirements

The tables below contain the requirements from the Reference Model to cover the Basic and High-Performance profiles. The table also includes a reference to the specification from Component Level Architecture and from Security Guidance to ensure traceability. If the related Specification does not exist, the reference will read “N/A” (and in bold “N/A” for mandatory requirements).

To ensure alignment with the infrastructure profile catalogue, the following requirements are referenced through:

  • Those relating to Cloud Infrastructure Software Profiles

  • Those relating to Cloud Infrastructure Hardware Profiles

  • Those relating to Cloud Infrastructure Management

  • Those relating to Cloud Infrastructure Security

2.3.1. Cloud Infrastructure Software Profile Capabilities

Table 2.1 Reference Model Requirements: Cloud Infrastructure Software Profile Capabilities

Reference Model Section

Reference

Description

Requirement for Basic Profile

Requirement for High-Performance Profile

Specification Reference

4.1.2

e.cap.001

Max number of vCPU that can be assigned to a single Pod by the Cloud Infrastructure

At least 16

At least 16

ra2.ch.011

4.1.2

e.cap.002

Max memory in MB that can be assigned to a single Pod by the Cloud Infrastructure

at least 32 GB

at least 32 GB

ra2.ch.012

4.1.2

e.cap.003

Max storage in GB that can be assigned to a single Pod by the Cloud Infrastructure

at least 320 GB

at least 320 GB

ra2.ch.010

4.1.2

e.cap.004

Max number of connection points that can be assigned to a single Pod by the Cloud Infrastructure

6

6

ra2.ntw.003

4.1.2

e.cap.005

Max storage in GB that can be attached / mounted to Pod by the Cloud Infrastructure

Up to 16TB (1)

Up to 16TB (1)

N/A

4.2.2

e.cap.006

CPU pinning support

Not required

Must support

ra2.k8s.009

4.2.2

e.cap.007

NUMA support

Not required

Must support

ra2.k8s.006

4.1.2

e.cap.008

IPSec Acceleration using the virtio-ipsec interface

Not required

Optional

N/A

4.1.2

e.cap.009

Crypto Acceleration using the virtio-crypto interface

Not required

Optional

N/A

4.1.2

e.cap.010

Transcoding Acceleration

Not required

Not required

N/A

4.1.2

e.cap.011

Programmable Acceleration

Not required

Not required

N/A

4.1.2

e.cap.012

Enhanced Cache Management: L=Lean; E=Equal; X=eXpanded

E

E

N/A

4.2.2

e.cap.013

SR-IOV over PCI-PT

Not required

Must support

ra2.ch.002 ra2.ch.003 ra2.k8s.007 ra2.ntw.004 ra2.ntw.008

4.1.2

e.cap.014

Hardware coprocessor support (GPU/NPU)

Not required

Not required

N/A

4.1.2

e.cap.015

SmartNICs

Not required

Optional

N/A

4.1.2

e.cap.016

FPGA/other Acceleration H/W

Not required

Optional

ra2.k8s.007 ra2.ntw.012

4.1.2

e.cap.017

Ability to monitor L2-L7 data from workload

n/a (2)

n/a (2)

N/A

4.1.4

i.cap.014

Specifies the proportion of CPU cores consumed by the Cloud Infrastructure system on the worker nodes. If SMT is used, it indicates the number of consumed SMT threads.

2

2

ra2.k8s.008

4.1.4

i.cap.015

Indicates the memory consumed by Cloud Infrastructure on the worker nodes

16 GB

16 GB

4.1.4

i.cap.016

Number of virtual cores per physical core; also known as CPU overbooking ratio that is required

01:01

01:01

ra2.ch.004 ra2.ch.005

4.1.4

i.cap.017

QoS enablement of the connection point (vNIC or interface)

Not required

Must support

N/A

4.1.4

i.cap.018

Support for huge pages

Not required

Must support

ra2.ch.001

4.1.4

i.pm.001

Monitor worker node CPU usage, per nanosecond

Must support

Must support

N/A

4.1.4

i.pm.002

Monitor pod CPU usage, per nanosecond

Must support

Must support

N/A

4.1.4

i.pm.003

Monitor worker node CPU utilisation (%)

Must support

Must support

N/A

4.1.4

i.pm.004

Monitor pod CPU utilisation

Must support

Must support

N/A

4.1.4

i.pm.005

Measure external storage IOPs

Must support

Must support

N/A

4.1.4

i.pm.006

Measure external storage throughput

Must support

Must support

N/A

4.1.4

i.pm.007

Measure external storage capacity

Must support

Must support

N/A

4.2.2

i.os.001

Host operating system must provide drivers etc. to support listed capabilities.

Must support

Must support

ra2.ch.004

Capabilities and Performance Measurements

(1) Defined in the .bronze configuration in RM section Storage Extensions

(2) In Kubernetes based infrastructures packet monitoring is out of the scope for the infrastructure.

2.3.2. Virtual Network Interface Specifications

The required number of connection points to a Pod is described in e.cap.004 above. This section describes the required bandwidth of those connection points.

Table 2.2 Reference Model Requirements: Network Interface Specifications

Reference Model Section

Reference

Description

Requirement for Basic Profile

Requirement for High-Performance Profile

Specification Reference

4.2.5

n1, n2, n3, n4, n5, n6

1, 2, 3, 4, 5, 6 Gbps

Must support

Must support

N/A

4.2.5

n10, n20, n30, n40, n50, n60

10, 20, 30, 40, 50, 60 Gbps

Must support

Must support

N/A

4.2.5

n25, n50, n75, n100, n125, n150

25, 50, 75, 100, 125, 150 Gbps

Must support

Must support

N/A

4.2.5

n50, n100 , n150, n200, n250 , n300

50, 100, 150, 200, 250, 300 Gbps

Must support

Must support

N/A

4.2.5

n100, n200, n300, n400, n500, n600

100, 200, 300, 400, 500, 600 Gbps

Must support

Must support

N/A

Virtual Network Interface Specifications

2.3.3. Cloud Infrastructure Software Profile Requirements

Table 2.3 Reference Model Requirements: Cloud Infrastructure Software Profile Requirements

Reference Model Section

Reference

Description

Requirement for Basic Profile

Requirement for High-Performance Profile

Specification Reference

5.1.1

infra.com. cfg.001

CPU allocation ratio

1:1

1:1

ra2.ch.005 ra2.ch.006

5.1.1

infra.com. cfg.002

NUMA awareness

Not required

Must support

ra2.k8s.006

5.1.1

infra.com. cfg.003

CPU pinning capability

Not required

Must support

ra2.k8s.009

5.1.1

infra.com. cfg.004

Huge pages

Not required

Must support

ra2.ch.001

5.1.2

infra.stg. cfg.002

Storage Block

Must support

Must support

ra2.stg.004

5.1.2

infra.stg. cfg.003

Storage with replication

Not required

Must support

N/A

5.1.2

infra.stg. cfg.004

Storage with encryption

Must support

Must support

N/A

5.1.2

infra.stg. acc.cfg.00 1

Storage IOPS oriented encryption

Not required

Must support

N/A

5.1.2

infra.stg. acc.cfg.00 2

Storage capacity oriented encryption

Not required

Not required

N/A

5.1.3

infra.net.cfg.001

IO virtualisation using virtio1.1

Must support (1)

Must support (1)

N/A

5.1.3

infra.net.cfg.002

The overlay network encapsulation protocol needs to enable ECMP in the underlay to take advantage of the scale-out features of the network fabric.(2)

Must support VXLAN, MPLSoUDP, GENEVE, other

No requirement specified

N/A

5.1.3

infra.net.cfg.003

Network Address Translation

Must support

Must support

N/A

5.1.3

infra.net.cfg.004

Security Groups

Must support

Must support

ra2.k8s.014

5.1.3

infra.net.cfg.005

SFC support

Not required

Must support

N/A

5.1.3

infra.net.cfg.006

Traffic patterns symmetry

Must support

Must support

N/A

5.1.3

infra.net.acc.cfg.00 1

vSwitch optimisation

Not required

Must support DPDK (3)

ra2.ntw.010

5.1.3

infra.net.acc.cfg.00 2

Support of HW offload

Not required

Optional, SmartNic

N/A

5.1.3

infra.net.acc.cfg.00 3

Crypto acceleration

Not required

Optional

N/A

5.1.3

infra.net.acc.cfg.00 4

Crypto Acceleration Interface

Not required

Optional

N/A

Virtual Networking

(1) Might have other interfaces (such as SR-IOV VFs to be directly passed to a VM or a Pod) or NIC-specific drivers on guest machines transiently allowed until more mature solutions are available with an acceptable level of efficiency to support telecom workloads (for example regarding CPU and energy consumption).

(2) In Kubernetes based infrastructures network separation is possible without an overlay (e.g.: with IPVLAN)

(3) This feature is not applicable for Kubernetes based infrastructures due to lack of vSwitch however workloads need access to user space networking solutions.

2.3.4. Cloud Infrastructure Hardware Profile Requirements

Table 2.4 Reference Model Requirements: Cloud Infrastructure Hardware Profile Requirements

Reference Model Section

Reference

Description

Requirement for Basic Profile

Requirement for High-Performance Profile

Specification Reference

5.4.1

infra.hw.cpu.cfg. 001

Minimum number of CPU sockets

2

2

ra2.ch.008

5.4.1

infra.hw.cpu.cfg. 002

Minimum number of Cores per CPU

20

20

ra2.ch.008

5.4.1

infra.hw.cpu.cfg. 003

NUMA Alignment

N

Y

ra2.ch.008

5.4.1

infra.hw.cpu.cfg. 004

Simultaneous Multithreading/ Symmetric Multiprocessing (SMT/SMP)

Must support

Optional

ra2.ch.004

5.4.1

infra.hw.cac.cfg. 001

GPU

Not required

Optional

N/A

5.4.2

infra.hw.stg.hdd. cfg.001

Local Storage HDD

No requirement specified

No requirement specified

N/A

5.4.2

infra.hw.stg.ssd. cfg.002

Local Storage SSD

Should support

Should support

ra2.ch.009

5.4.3

infra.hw.nic.cfg. 001

Total Number of NIC Ports available in the host

4

4

ra2.ch.013

5.4.3

infra.hw.nic.cfg. 002

Port speed specified in Gbps (minimum values)

10

25

ra2.ch.014 ra2.ch.015

5.4.3

infra.hw.pci.cfg. 001

Number of PCIe slots available in the host

8

8

ra2.ch.016

5.4.3

infra.hw.pci.cfg. 002

PCIe speed

Gen 3

Gen 3

ra2.ch.016

5.4.3

infra.hw.pci.cfg. 003

PCIe Lanes

8

8

ra2.ch.016

5.4.3

infra.hw.nac.cfg. 001

Cryptographic Acceleration

Not required

Optional

N/A

5.4.3

infra.hw.nac.cfg. 002

A SmartNIC that is used to offload vSwitch functionality to hardware

Not required

Optional (1)

N/A

5.4.3

infra.hw.nac.cfg. 003

Compression

Optional

Optional

N/A

Network Acceleration Configurations

(1) There is no vSwitch in case of containers, but a SmartNIC can be used to offload any other network processing.

2.3.5. Edge Cloud Infrastructure Hardware Profile Requirements

In the case of Telco Edge Cloud Deployments, hardware requirements can differ from the above to account for environmental and other constraints. The Reference Model Hybrid Multi-Cloud Architecture includes considerations specific to deployments at the edge of the network. The infrastructure profiles “Basic” and “High Performance” as per Profiles and Workload Flavours still apply, but a number of requirements of the above table are relaxed as follows:

Table 2.5 Reference Model Requirements: Edge Cloud Infrastructure Hardware Profile Requirements

Reference Model Section

Reference

Description

Requirement for Basic Profile

Requirement for High-Performance Profile

Specification Reference

8.x.x

infra.hw.cpu.cfg.001

sockets

8.x.x

infra.hw.cpu.cfg.002

Minimum number of Cores per CPU

1

1

ra2.ch.008

8.x.x

infra.hw.cpu.cfg.003

NUMA Alignment

N

Y (1)

ra2.ch.008

Telco Edge Cloud: Infrastructure Profiles.

(1) immaterial if the number of CPU sockets (infra.hw.cpu.cfg.001) is 1.

2.3.6. Cloud Infrastructure Management Requirements

Table 2.6 Reference Model Requirements: Cloud Infrastructure Management Requirements

Reference Model Section

Reference

Description

Requirement (common to all Profiles)

Specification Reference

4.1.5

e.man.001

Capability to allocate virtual compute resources to a workload

Must support

N/A

4.1.5

e.man.002

Capability to allocate virtual storage resources to a workload

Must support

N/A

4.1.5

e.man.003

Capability to allocate virtual networking resources to a workload

Must support

N/A

4.1.5

e.man.004

Capability to isolate resources between tenants

Must support

N/A

4.1.5

e.man.005

Capability to manage workload software images

Must support

N/A

4.1.5

e.man.006

Capability to provide information related to allocated virtualised resources per tenant

Must support

N/A

4.1.5

e.man.007

Capability to notify state changes of allocated resources

Must support

N/A

4.1.5

e.man.008

Capability to collect and expose performance information on virtualised resources allocated

Must support

N/A

4.1.5

e.man.009

Capability to collect and notify fault information on virtualised resources

Must support

N/A

Cloud Infrastructure Management Capabilities.

2.3.7. Cloud Infrastructure Security Requirements

Table 2.7 Reference Model Requirements: Cloud Infrastructure Security Requirements

Reference Model Section

Reference

Description

Specification Reference

7.9.1

sec.gen.001

The Platform must maintain the specified configuration.

7.9.1

sec.gen.002

All systems part of Cloud Infrastructure must support password hardening as defined in CIS Password Policy Guide. Hardening: CIS Password Policy Guide

5.3.1 Node Hardening: Securing Kubernetes Hosts

7.9.1

sec.gen.003

All servers part of Cloud Infrastructure must support a root of trust and secure boot.

7.9.1

sec.gen.004

The Operating Systems of all the servers part of Cloud Infrastructure must be hardened by removing or disabling unnecessary services, applications and network protocols, configuring operating system user authentication, configuring resource controls, installing and configuring additional security controls where needed, and testing the security of the Operating System. (NIST SP 800-123)

5.2 Principles and 5.3 Node Hardening

7.9.1

sec.gen.005

The Platform must support Operating System level access control

5.3 Node Hardening

7.9.1

sec.gen.006

The Platform must support Secure logging. Logging with root account must be prohibited when root privileges are not required.

5.3.2 Restrict direct access to nodes

7.9.1

sec.gen.007

All servers part of Cloud Infrastructure must be Time synchronized with authenticated Time service.

7.9.1

sec.gen.008

All servers part of Cloud Infrastructure must be regularly updated to address security vulnerabilities.

5.3.3 Vulnerability assessment

7.9.1

sec.gen.009

The Platform must support Software integrity protection and verification and must scan source code and manifests.

5.4 Securing Kubernetes orchestrator

7.9.1

sec.gen.010

The Cloud Infrastructure must support encrypted storage, for example, block, object and file storage, with access to encryption keys restricted based on a need to know. Controlled Access Based on the Need to Know

7.9.1

sec.gen.011

The Cloud Infrastructure should support Read and Write only storage partitions (write only permission to one or more authorized actors).

7.9.1

sec.gen.012

The Operator must ensure that only authorized actors have physical access to the underlying infrastructure.

7.9.1

sec.gen.013

The Platform must ensure that only authorized actors have logical access to the underlying infrastructure.

5.4 Securing Kubernetes orchestrator

7.9.1

sec.gen.014

All servers part of Cloud Infrastructure should support measured boot and an attestation server that monitors the measurements of the servers.

7.9.1

sec.gen.015

Any change to the Platform must be logged as a security event, and the logged event must include the identity of the entity making the change, the change, the date and the time of the change.

7.9.2

sec.sys.001

The Platform must support authenticated and secure access to API, GUI and command line interfaces.

5.4 Securing Kubernetes orchestrator

7.9.2

sec.sys.002

The Platform must support Traffic Filtering for workloads (for example, Firewall).

7.9.2

sec.sys.003

The Platform must support Secure and encrypted communications, and confidentiality and integrity of network traffic.

5.4.3 Use Transport Layer Security and Service Mesh

7.9.2

sec.sys.004

The Cloud Infrastructure must support authentication, integrity and confidentiality on all network channels.

5.4.3 Use Transport Layer Security and Service Mesh

7.9.2

sec.sys.005

The Cloud Infrastructure must segregate the underlay and overlay networks.

7.9.2

sec.sys.006

The Cloud Infrastructure must be able to utilise the Cloud Infrastructure Manager identity lifecycle management capabilities.

5.2 Principles

7.9.2

sec.sys.007

The Platform must implement controls enforcing separation of duties and privileges, least privilege use and least common mechanism (Role-Based Access Control).

5.2 Principles 5.4 Securing Kubernetes orchestrator

7.9.2

sec.sys.008

The Platform must be able to assign the Entities that comprise the tenant networks to different trust domains. Communication between different trust domains is not allowed, by default.

7.9.2

sec.sys.009

The Platform must support creation of Trust Relationships between trust domains.

7.9.2

sec.sys.010

For two or more domains without existing trust relationships, the Platform must not allow the effect of an attack on one domain to impact the other domains either directly or indirectly.

7.9.2

sec.sys.011

The Platform must not reuse the same authentication credential (e.g., key-pair) on different Platform components (e.g., on different hosts, or different services).

7.9.2

sec.sys.012

The Platform must protect all secrets by using strong encryption techniques, and storing the protected secrets externally from the component

7.9.2

sec.sys.013

The Platform must provide secrets dynamically as and when needed.

7.9.2

sec.sys.014

The Platform should use Linux Security Modules such as SELinux to control access to resources.

7.9.2

sec.sys.015

The Platform must not contain back door entries (unpublished access points, APIs, etc.).

7.9.2

sec.sys.016

Login access to the platform’s components must be through encrypted protocols such as SSH v2 or TLS v1.2 or higher. Note: Hardened jump servers isolated from external networks are recommended

5.4 Securing Kubernetes orchestrator

7.9.2

sec.sys.017

The Platform must provide the capability of using digital certificates that comply with X.509 standards issued by a trusted

7.9.2

sec.sys.018

The Platform must provide the capability of allowing certificate renewal and revocation.

7.9.2

sec.sys.019

The Platform must provide the capability of testing the validity of a digital certificate (CA signature, validity period, non revocation, identity).

7.9.2

sec.sys.020

The Cloud Infrastructure architecture should rely on Zero Trust principles to build a secure by design environment.

7.9.3

sec.ci.001

The Platform must support Confidentiality and Integrity of data at rest and in-transit. by design environment.

5.4 Securing Kubernetes orchestrator

7.9.3

sec.ci.002

The Platform should support self-encrypting storage devices. data at rest and in-transit. by design environment.

7.9.3

sec.ci.003

The Platform must support Confidentiality and Integrity of data related metadata.

7.9.3

sec.ci.004

The Platform must support Confidentiality of processes and restrict information sharing with only the process owner (e.g., tenant).

7.9.3

sec.ci.005

The Platform must support Confidentiality and Integrity of process-related metadata and restrict information sharing with only the process owner (e.g., tenant).

7.9.3

sec.ci.006

The Platform must support Confidentiality and Integrity of workload resource utilization (RAM, CPU,

Storage, Network I/O, cache, hardware offload) and restrict information sharing with only the workload owner (e.g., tenant).

7.9.3

sec.ci.007

The Platform must not allow Memory Inspection by any actor other than the authorized actors for the Entity to which Memory is assigned (e.g., tenants owning the workload), for Lawful Inspection, and by secure monitoring services.

7.9.3

sec.ci.008

The Cloud Infrastructure must support tenant networks segregation.

5.7 Create and define Network Policies

7.9.3

sec.ci.009

For sensitive data encryption, the key management service should leverage a Hardware Security Module to manage and protect cryptographic keys.

7.9.4

sec.wl.001

The Platform must support Workload placement policy.

7.9.4

sec.wl.002

The Cloud Infrastructure must provide methods to ensure the platform’s trust status and integrity (e.g. remote attestation, Trusted Platform Module).

7.9.4

sec.wl.003

The Platform must support secure provisioning of workloads.

5.4 Securing Kubernetes orchestrator

7.9.4

sec.wl.004

The Platform must support Location assertion (for mandated in-country or location requirements).

7.9.4

sec.wl.005

The Platform must support the separation of production and non-production Workloads.

5.4 Securing Kubernetes orchestrator

7.9.4

sec.wl.006

The Platform must support the separation of Workloads based on their categorisation (for example, payment card information, healthcare, etc.).

5.4 Securing Kubernetes orchestrator

7.9.4

sec.wl.007

The Operator must implement processes and tools to verify VNF authenticity and integrity.

5.13 Trusted Registry

7.9.5

sec.img.001

Images from untrusted sources must not be used.

5.13 Trusted Registry

7.9.5

sec.img.002

Images must be scanned to be maintained free from known vulnerabilities.

5.13 Trusted Registry

7.9.5

sec.img.003

Images must not be configured to run with privileges higher than the privileges of the actor authorized to run them.

5.11 Run-Time Security

7.9.5

sec.img.004

Images must only be accessible to authorized actors.

7.9.5

sec.img.005

Image Registries must only be accessible to authorized actors.

7.9.5

sec.img.006

Image Registries must only be accessible over secure networks that enforce authentication, integrity and confidentiality.

5.13 Trusted Registry

7.9.5

sec.img.007

Image registries must be clear of vulnerable and out of date versions.

5.13 Trusted Registry

7.9.5

sec.img.008

Images must not include any secrets. Secrets include passwords, cloud provider credentials, SSH keys, TLS certificate keys, etc.

5.12 Secrets Management

7.9.5

sec.img.009

CIS Hardened Images should be used whenever possible.

7.9.5

sec.img.010

Minimalist base images should be used whenever possible.

7.9.6

sec.lcm.001

The Platform must support Secure Provisioning, Availability, and Deprovisioning (Secure Clean-Up) of workload resources where Secure Clean-Up includes tear-down, defense against virus or other attacks.

7.9.6

sec.lcm.002

Cloud operations staff and systems must use management protocols limiting security risk such as SNMPv3, SSH v2, ICMP, NTP, syslog and TLS v1.2 or higher.

5.4 Securing Kubernetes orchestrator

7.9.6

sec.lcm.003

The Cloud Operator must implement and strictly follow change management processes for Cloud Infrastructure, Cloud Infrastructure Manager and other components of the cloud, and Platform change control on hardware.

7.9.6

sec.lcm.004

The Cloud Operator should support automated templated approved changes.

7.9.6

sec.lcm.005

Platform must provide logs and these logs must be regularly monitored for anomalous behavior.

5.10 Enable Logging and Monitoring

7.9.6

sec.lcm.006

The Platform must verify the integrity of all Resource management requests.

7.9.6

sec.lcm.007

The Platform must be able to update newly instantiated, suspended, hibernated, migrated and restarted images with current time information.

5.4 Securing Kubernetes orchestrator

7.9.6

sec.lcm.008

The Platform must be able to update newly instantiated, suspended, hibernated, migrated and restarted images with relevant DNS information.

7.9.6

sec.lcm.009

The Platform must be able to update the tag of newly instantiated, suspended, hibernated, migrated and restarted images with relevant geolocation (geographical) information.

7.9.6

sec.lcm.010

The Platform must log all changes to geolocation along with the mechanisms and sources of location information (i.e. GPS, IP block, and timing).

7.9.6

sec.lcm.011

The Platform must implement Security life cycle management processes including the proactive update and patching of all deployed Cloud Infrastructure software.

7.9.6

sec.lcm.012

The Platform must log any access privilege escalation.

7.9.7

sec.mon.001

Platform must provide logs and these logs must be regularly monitored for events of interest. The logs must contain the following fields: event type, date/time, protocol, service or program used for access, success/failure, login ID or process ID, IP address and ports (source and destination) involved.

7.9.7

sec.mon.002

Security logs must be time synchronised.

7.9.7

sec.mon.003

The Platform must log all changes to time server source, time, date and time zones.

7.9.7

sec.mon.004

The Platform must secure and protect Audit logs (containing sensitive information) both in-transit and at rest.

7.9.7

sec.mon.005

The Platform must Monitor and Audit various behaviours of connection and login attempts to detect access attacks and potential access attempts and take corrective actions accordingly.

7.9.7

sec.mon.006

The Platform must Monitor and Audit operations by authorized account access after login to detect malicious operational activity and take corrective actions accordingly.

7.9.7

sec.mon.007

The Platform must Monitor and Audit security parameter configurations for compliance with defined security policies.

7.9.7

sec.mon.008

The Platform must Monitor and Audit externally exposed interfaces for illegal access (attacks) and take corrective security hardening measures.

7.9.7

sec.mon.009

The Platform must Monitor and Audit service handling for various attacks (malformed messages, signalling flooding and replaying, etc.) and take corrective actions accordingly.

7.9.7

sec.mon.010

The Platform must Monitor and Audit running processes to detect unexpected or unauthorized processes and take corrective actions accordingly.

7.9.7

sec.mon.011

The Platform must Monitor and Audit logs from infrastructure elements and workloads to detected anomalies in the system components and take corrective actions accordingly.

7.9.7

sec.mon.012

The Platform must Monitor and Audit Traffic patterns and volumes to prevent malware download attempts.

7.9.7

sec.mon.013

The monitoring system must not affect the security (integrity and confidentiality) of the infrastructure, workloads, or the user data (through back door entries).

7.9.7

sec.mon.014

The Monitoring systems should not impact IAAS, PAAS, and SAAS SLAs including availability SLAs.

7.9.7

sec.mon.015

The Platform must ensure that the Monitoring systems are never starved of resources and must activate alarms when resource utilisation exceeds a configurable threshold.

7.9.7

sec.mon.016

The Platform Monitoring components should follow security best practices for auditing, including secure logging and tracing.

7.9.7

sec.mon.017

The Platform must audit systems for any missing security patches and take appropriate actions.

5.3.3 Vulnerability assessment

7.9.7

sec.mon.018

The Platform, starting from initialization, must collect and analyze logs to identify security events, and store these events in an external system.

5.3.4 Patch management

7.9.7

sec.mon.019

The Platform’s components must not include an authentication credential, e.g., password, in any logs, even if encrypted.

7.9.7

sec.mon.020

The Platform’s logging system must support the storage of security audit logs for a configurable period of time.

7.9.7

sec.mon.021

The Platform must store security events locally if the external logging system is unavailable and shall periodically attempt to send these to the external logging system until successful.

7.9.8

sec.oss.001

Open source code must be inspected by tools with various capabilities for static and dynamic code analysis.

5.3.3 Vulnerability assessment

7.9.8

sec.oss.002

The CVE (Common Vulnerabilities and Exposures) must be used to identify vulnerabilities and their severity rating for open source code part of Cloud Infrastructure and workloads software.

7.9.8

sec.oss.003

Critical and high severity rated vulnerabilities must be fixed in a timely manner. Refer to the CVSS (Common Vulnerability Scoring System to know a vulnerability score and its associated rate (low, medium, high, or critical).

7.9.8

sec.oss.004

A dedicated internal isolated repository separated from the production environment must be used to store vetted open source content.

5.13 Trusted Registry

7.9.8

sec.oss.005

A Software Bill of Materials (SBOM) should be provided or build, and maintained to identify the software components and their origins.

7.9.9

sec.arch.001

Threat Modelling methodologies and tools should be used during the Secure Design and Architecture stage triggered by Software Feature Design trigger. It may be done manually or using tools like open source OWASP Threat Dragon

7.9.9

sec.arch.002

Security Control Baseline Assessment should be performed during the Secure Design and Architecture stage triggered by Software Feature Design trigger. Typically done manually by internal or independent assessors.

7.9.10

sec.code.001

SAST -Static Application Security Testing must be applied during Secure Coding stage triggered by Pull, Clone or Comment trigger. Security testing that analyses application source code for software vulnerabilities and gaps against best practices. Example: open source OWASP range of tools.

7.9.10

sec.code.002

SCA - Software Composition Analysis should be applied during Secure Coding stage triggered by Pull, Clone or Comment trigger. Security testing that analyses application source code or compiled code for software components with known vulnerabilities. Example: open source OWASP range of tools.

7.9.10

sec.code.003

Source Code Review should be performed continuously during Secure Coding stage. Typically done manually.

7.9.10

sec.code.004

Integrated SAST via IDE Plugins should be used during Secure Coding stage triggered by Developer Code trigger. On the local machine: through the IDE or integrated test suites; triggered on completion of coding be developer.

7.9.10

sec.code.005

SAST of Source Code Repo should be performed during Secure Coding stage triggered by Developer Code trigger. Continuous delivery pre-deployment: scanning prior to deployment.

7.9.11

sec.bld.001

SAST -Static Application Security Testing should be applied during the Continuous Build, Integration and Testing stage triggered by Build and Integrate trigger. Example: open source OWASP range of tools.

7.9.11

sec.bld.002

SCA - Software Composition Analysis should be applied during the Continuous Build, Integration and Testing stage triggered by Build and Integrate trigger. Example: open source OWASP range of tools.

7.9.11

sec.bld.003

Image Scan must be applied during the Continuous Build, Integration and Testing stage triggered by Package trigger. Example: A push of a container image to a container registry may trigger a vulnerability scan before the image becomes available in the registry.

7.9.11

sec.bld.004

DAST - Dynamic Application Security Testing should be applied during the Continuous Build, Integration and Testing stage triggered by Stage & Test trigger. Security testing that analyses a running application by exercising application functionality and detecting vulnerabilities based on application behaviour and response. Example: OWASP ZAP.

7.9.11

sec.bld.005

Fuzzing should be applied during the Continuous Build, Integration and testing stage triggered by Stage & Test trigger. Fuzzing or fuzz testing is an automated software testing technique that involves providing invalid, unexpected, or random data as inputs to a computer program. Example: GitLab Open Sources Protocol Fuzzer Community Edition.

7.9.11

sec.bld.006

IAST - Interactive Application Security Testing should be applied during the Continuous Build, Integration and Testing stage triggered by Stage & Test trigger. Software component deployed with an application that assesses application behaviour and detects presence of vulnerabilities on an application being exercised in realistic testing scenarios. Example: Contrast Community Edition.

7.9.12

sec.del.001

Image Scan must be applied during the Continuous Delivery and Deployment stage triggered by Publish to Artifact and Image Repository trigger. Example: GitLab uses the open-source Clair engine for container image scanning.

7.9.12

sec.del.002

Code Signing must be applied during the Continuous Delivery and Deployment stage triggered by Publish to Artifact and Image Repository trigger. Code Signing provides authentication to assure that downloaded files are from the publisher named on the certificate.

7.9.12

sec.del.003

Artifact and Image Repository Scan should be continuously applied during the Continuous Delivery and Deployment stage. Example: GitLab uses the open source Clair engine for container scanning.

7.9.12

sec.del.004

Component Vulnerability Scan must be applied during the Continuous Delivery and Deployment stage triggered by Instantiate Infrastructure trigger. The vulnerability scanning system is deployed on the cloud platform to detect security vulnerabilities of specified components through scanning and to provide timely security protection. Example: OWASP Zed Attack Proxy (ZAP).

7.9.13

sec.run.001

Component Vulnerability Monitoring must be continuously applied during the Runtime Defence and Monitoring stage and remediation actions must be applied for high severity rated vulnerabilities. Security technology that monitors components like virtual servers and assesses data, applications, and infrastructure for security risks.

7.9.13

sec.run.002

RASP - Runtime Application Self- Protection should be continuously applied during the Runtime Defence and Monitoring stage. Security technology deployed within the target application in production for detecting, alerting, and blocking attacks.

7.9.13

sec.run.003

Application testing and Fuzzing should be continuously applied during the Runtime Defence and Monitoring stage. Fuzzing or fuzz testing is an automated software testing technique that involves providing invalid, unexpected, or random data as inputs to a computer program. Example: GitLab Open Sources Protocol Fuzzer Community Edition.

7.9.13

sec.run.004

Penetration Testing should be continuously applied during the Runtime Defence and Monitoring stage. Typically done manually.

7.9.14

sec.std.001

The Cloud Operator should comply with Center for Internet Security CIS Controls (https://www.cisecur ity.org/)

7.9.14

sec.std.002

The Cloud Operator, Platform and Workloads should follow the guidance in the CSA Security Guidance for Critical Areas of Focus in Cloud Computing (latest version) https://cloudsecurityalliance. org/

7.9.14

sec.std.003

The Platform and Workloads should follow the guidance in the OWASP Cheat Sheet Series (OCSS)

7.9.14

sec.std.004

The Cloud Operator, Platform and Workloads should ensure that their code is not vulnerable to the OWASP Top Ten Security Risks https://owasp.org/www-project-top-t en/

7.9.14

sec.std.005

The Cloud Operator, Platform and Workloads should strive to improve their maturity on the OWASP Software Maturity Model (SAMM)

7.9.14

sec.std.006

The Cloud Operator, Platform and Workloads should utilize the OW ASP Web Security Testing Guide

7.9.14

sec.std.007

The Cloud Operator, and Platform should satisfy the requirements for Information Management Systems specified in ISO/IEC 27001. ISO/IEC 27002:2013 - ISO/IEC 27001 is the international Standard for best-practice information security management systems (ISMSs).

7.9.14

sec.std.008

The Cloud Operator, and Platform should implement the Code of practice for Security Controls specified ISO/IEC 27002:2013 (or la test)

7.9.14

sec.std.009

The Cloud Operator, and Platform should implement the ISO/IEC 27 032:2012 (or latest) Guidelines for Cybersecurity techniques. ISO/IEC 27032 - ISO/IEC 27032 is the international Standard focusing explicitly on cybersecurity.

7.9.14

sec.std.010

The Cloud Operator should conform to the ISO/IEC 27035 standard for incidence management. ISO/IEC 27035 - ISO/IEC 27035 is the international Standard for incident management.

7.9.14

sec.std.011

The Cloud Operator should conform to the ISO/IEC 27031 standard for business continuity. ISO/IEC 27031 - ISO/IEC 27031 is the international Standard for ICT readiness for business continuity.

7.9.14

sec.std.012

The Public Cloud Operator must, and the Private Cloud Operator may be certified to be compliant with the International Standard on Awareness Engagements (ISAE) 3402 (in the US: SSAE 16). International Standard on Awareness Engagements (ISAE) 3402. US Equivalent: SSAE16.

Consolidated Security Requirements

2.4. Kubernetes Architecture Requirements

The requirements in this section are to be delivered in addition to those in section 2.2, and have been created to support the Principles defined in the Overview of this Reference Architecture.

The Reference Model (RM) defines the Cloud Infrastructure, which consists of the physical resources, virtualised resources and a software management system.

In virtualisation platforms, the Cloud Infrastructure consists of the Guest Operating System, Hypervisor and, if needed, other software such as libvirt. The Cloud Infrastructure Management component is responsible for, among others, tenant management, resources management, inventory, scheduling, and access management.

With regards to containerisation platforms, the scope of the following Architecture requirements include the Cloud Infrastructure Hardware (e.g. physical resources), Cloud Infrastructure Software (e.g. Hypervisor (optional), Container Runtime, virtual or container Orchestrator(s), Operating System), and infrastructure resources consumed by virtual machines or containers.

Table 2.8 Kubernetes Architecture Requirements

Reference

Category

Sub-category

Description

Specification Reference

gen.cnt.02

General

Cloud nativeness

The Architecture must support immutable infrastructure.

ra2.ch.017

gen.cnt.03

General

Cloud nativeness

The Architecture must run conformant Kubernetes as defined by the CNCF.

ra2.k8s.001

gen.cnt.04

General

Cloud nativeness

The Architecture must support clearly defined abstraction layers.

gen.cnt.05

General

Cloud nativeness

The Architecture should support configuration of all components in an automated manner using openly published API definitions.

gen.scl.01

General

Scalability

The Architecture should support policy driven horizontal auto-scaling of workloads.

gen.rsl.01

General

Resiliency

The Architecture must support resilient Kubernetes components that are required for the continued availability of running workloads.

ra2.k8s.004

gen.rsl.02

General

Resiliency

The Architecture should support resilient Kubernetes service components that are not subject to gen.rsl.01.

ra2.k8s.002, ra2.k8s.003

gen.avl.01

General

Availability

The Architecture must provide High Availability for Kubernetes components.

ra2.k8s.002, ra2.k8s.003, ra2.k8s.004

gen.ost.01

Openness

Availability

The Architecture should embrace open-based standards and technologies.

ra2.crt.001, ra2.crt.002, ra2.ntw.002, ra2.ntw.006, ra2.ntw.007

inf.com.01

Infrastructure

Compute

The Architecture must provide compute resources for Pods. technologies.

ra2.k8s.004

inf.stg.01

Infrastructure

Storage

The Architecture must support the ability for an operator to choose whether or not to deploy persistent storage for Pods.

ra2.stg.004

inf.ntw.01

Infrastructure

Network

The Architecture must support network resiliency on the Kubernetes nodes.

inf.ntw.02

Infrastructure

Network

The Architecture must support fully redundant network connectivity to the Kubernetes nodes, leveraging multiple network connections.

inf.ntw.03

Infrastructure

Network

The networking solution should be able to be centrally administrated and configured.

ra2.ntw.001, ra2.ntw.004

inf.ntw.04

Infrastructure

Network

The Architecture must support dual stack IPv4 and IPv6 for Kubernetes workloads.

ra2.ch.007, ra2.k8s.010

inf.ntw.05

Infrastructure

Network

The Architecture must support capabilities for integrating SDN controllers.

inf.ntw.06

Infrastructure

Network

The Architecture must support more than one networking solution.

ra2.ntw.005, ra2.ntw.007

inf.ntw.07

Infrastructure

Network

The Architecture must support the ability for an operator to choose whether or not to deploy more than one networking solution.

ra2.ntw.005

inf.ntw.08

Infrastructure

Network

The Architecture must provide a default network which implements the Kubernetes network model.

ra2.ntw.002

inf.ntw.09

Infrastructure

Network

The networking solution must not interfere with or cause interference to any interface or network it does not own.

inf.ntw.10

Infrastructure

Network

The Architecture must support Cluster wide coordination of IP address assignment.

inf.ntw.13

Infrastructure

Network

The platform must allow specifying multiple separate IP pools. Tenants are required to select at least one IP pool that is different from the control infrastructure IP pool or other tenant IP pools.

inf.ntw.14

Infrastructure

Network

The platform must allow NATless traffic (i.e. exposing the pod IP address directly to the outside), allowing source and destination IP addresses to be preserved in the traffic headers from workloads to external networks. This is needed e.g. for signaling applications, using SIP and Diameter protocols.

ra2.ntw.011

inf.ntw.15

Infrastructure

Network

The platform must support LoadBalancer Publishing Service (ServiceType)

inf.ntw.16

Infrastructure

Network

The platform must support Ingress.

inf.ntw.17

Infrastructure

Network

The platform should support NodePort Publishing Service (ServiceTypes).

inf.ntw.18

Infrastructure

Network

The platform should support ExternalName Publishing Service (ServiceTypes).

inf.vir.01

Infrastructure

Virtual Infr astructure

The Architecture must support the capability for Containers to consume infrastructure resources abstracted by Host Operating Systems that are running within a virtual machine.

ra2.ch.005 ra2.ch.011

inf.phy.01

Infrastructure

Physical Infrastructu re

The Architecture must support the capability for Containers to consume infrastructure resources abstracted by Host Operating Systems that are running within a physical server.

ra2.ch.008

kcm.gen.01

Kubernetes Cluster

General

The Architecture must support policy driven horizontal auto- scaling of Kubernetes Cluster.

N/A

kcm.gen.02

Kubernetes Cluster

General

The Architecture must enable workload resiliency.

ra2.k8s.004

int.api.01

API

General

The Architecture must leverage the Kubernetes APIs to discover and declaratively manage compute (virtual and bare metal resources), network, and storage.

For Networking: ra2.ntw.001, ra2.ntw.008, ra2.app.006. Compute/storage not yet met.

int.api.02

API

General

The Architecture must support the usage of a Kubernetes Application package manager using the Kubernetes API, like Helm v3. network, and storage.

ra2.pkg.001

int.api.03

API

General

The Architecture must support stable features in its APIs.

int.api.04

API

General

The Architecture must support limited backward compatibility in its APIs. Support for the whole API must not be dropped, but the schema or other details can change.