9. Gaps, Innovation, and Development

The purpose of this chapter is to identify the gaps between what is required for automated deployment of VNFs on Cloud Infrastructure frameworks and the framework offered by OpenStack. Once gaps are identified, the next step will be to propose a plan to address these gaps. The most obvious way to address the gaps will be to propose a set of APIs in the upstream OpenStack community

9.1. The Gap

9.1.1. Autoscaling

With regards to resource autoscaling (gen.scl.01 General Recommendations) it is recommended that the NFVO/VNFM manages the policy and triggers a scale-up or scale-down action based on application telemetry, event, AI, or ML etc. While the use of telemetry and alarming system can trigger a scaling operation based on resource utilisation, without application context this may not provide the granularity or reaction time required by the application. It is therefore suggested that an OpenStack scaling operation is called using an appropriate autoscaling web-hook by the NFVO/VNFM.

For more information on auto-scaling with Heat please see the OpenStack document “Autoscaling with heat [201]”. Please note that the OpenStack Senlin service is still under development with major architectural changes made in the OpenStack Ussuri release.

Please note: physical compute node autoscaling is out of scope.

1

Cloud infrastructure reference model. GSMA PRD NG.126 v3.0, 2022.

2

Openstack documentation. URL: https://docs.openstack.org/.

3

Network functions virtualisation (nfv); infrastructure overview. ETSI GS NFV-INF 001 V1.1.1, January 2015. URL: https://www.etsi.org/deliver/etsi_gs/NFV-INF/001_099/001/01.01.01_60/gs_NFV-INF001v010101p.pdf.

4

Openstack use cases. URL: https://docs.openstack.org/arch-design/use-cases.html.

5

Open vswitch. URL: https://www.openvswitch.org/.

6

OpenStack Community. Openstack wallaby projects. URL: https://docs.openstack.org/wallaby/projects.html.

7

Scott O. Bradner. Key words for use in rfcs to indicate requirement levels. RFC 2119, March 1997. URL: https://www.rfc-editor.org/info/rfc2119, doi:10.17487/RFC2119.

8

H. Philip White. Center for internet security - password policy guide. 2020. URL: https://www.cisecurity.org/insights/white-papers/cis-password-policy-guide.

9

Center for internet security - controls v7.1. URL: https://www.cisecurity.org/controls/cis-controls-list.

10

Openstack - cpu dedicated set. URL: https://docs.openstack.org/nova/latest/configuration/config.html#compute.cpu_dedicated_set.

11

Openstack - cpu topologies. URL: https://docs.openstack.org/nova/latest/admin/cpu-topologies.html.

12

Openstack - neutron plugins and drivers. URL: https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers.

13

Openstack - tags. URL: https://specs.openstack.org/openstack/api-wg/guidelines/tags.html.

14

Openstack - configuring the stateful services. URL: https://docs.openstack.org/ha-guide/control-plane-stateful.html.

15

Openstack - senlin documentation. URL: https://docs.openstack.org/senlin/wallaby/.

16

Openstack - neutron ovs agent support for baremetal with smart nic. URL: https://specs.openstack.org/openstack/neutron-specs/specs/stein/neutron-ovs-agent-support-baremetal-with-smart-nic.html.

17

National telecommunications and information administration - software bill of materials. URL: https://www.ntia.gov/SBOM.

18

Center for internet security. URL: https://www.cisecurity.org/.

19

Cloud security alliance. URL: https://cloudsecurityalliance.org/.

20

Open web application security project - cheat sheet series. URL: https://github.com/OWASP/CheatSheetSeries.

21

Open web application security project. URL: https://www.owasp.org.

22

Open web application security project - top ten security risks. URL: https://owasp.org/www-project-top-ten/.

23

Open web application security project - software maturity model (samm). URL: https://owaspsamm.org/blog/2019/12/20/version2-community-release/.

24

Open web application security project - web security testing guide. URL: https://github.com/OWASP/wstg/tree/master/document.

25

Iso (international organization for standardization) and iec (international electrotechnical commission) iso/iec 27001:2013. 2013. URL: https://www.iso.org/obp/ui/#iso:std:iso-iec:27001:ed-2:v1:en.

26

Iso (international organization for standardization) and iec (international electrotechnical commission) iso/iec 27002:2013. 2013. URL: https://www.iso.org/obp/ui/#iso:std:iso-iec:27002:ed-2:v1:en.

27

Iso (international organization for standardization) and iec (international electrotechnical commission) iso/iec 7032:2012. 2012. URL: https://www.iso.org/obp/ui/#iso:std:iso-iec:27032:ed-1:v1:en.

28

Openstack storage. URL: https://docs.openstack.org/arch-design/design-storage/design-storage-concepts.html#table-openstack-storage.

29

Openstack cinder driver support matrix. URL: https://docs.openstack.org/cinder/latest/reference/support-matrix.html.

30

Tungsten fabric - multicloud multistack sdn. URL: https://tungsten.io.

31

Openstack glossary. URL: https://docs.openstack.org/doc-contrib-guide/common/glossary.html.

32

Openstack feature support matrix. URL: https://docs.openstack.org/nova/latest/user/support-matrix.html.

33

Openstack storage architecture design. URL: https://docs.openstack.org/arch-design/design-storage.html.

34

Openstack nova: kvm. URL: https://docs.openstack.org/nova/wallaby/admin/configuration/hypervisor-kvm.html.

35

Openstack - hardening the virtualization layers. URL: https://docs.openstack.org/security-guide/compute/hardening-the-virtualization-layers.html.

36

Openstack reference architecture for 100, 300 and 500 nodes. URL: https://fuel-ccp.readthedocs.io/en/latest/design/ref_arch_100_nodes.html.

37

(dpdk) release notes. URL: http://doc.dpdk.org/guides/rel_notes.

38

(dpdk) performance reports. URL: http://core.dpdk.org/perf-reports/.

39

Robert Moskowitz, Daniel Karrenberg, Yakov Rekhter, Eliot Lear, and Geert Jan de Groot. Address allocation for private internets. RFC 1918, February 1996. URL: https://www.rfc-editor.org/info/rfc1918, doi:10.17487/RFC1918.

40

Openstack - introducing octavia. URL: https://docs.openstack.org/octavia/latest/reference/introduction.html.

41

Openstack octavia (load-balancer service). URL: https://governance.openstack.org/tc/reference/projects/octavia.html.

42

Openstack/neutron-vpnaas. URL: https://opendev.org/openstack/neutron-vpnaas.

43

Openstack neutron: plugins. URL: https://wiki.openstack.org/wiki/Neutron#Plugins.

44

Openstack neutron: api extensions. URL: https://docs.openstack.org/neutron/latest/contributor/internals/api_extensions.html.

45

Openstack networking api v2.0: list extensions. URL: https://docs.openstack.org/api-ref/network/v2/#list-extensions.

46

Openstack networking api v2.0: show extension details. URL: https://docs.openstack.org/api-ref/network/v2/#show-extension-details.

47

Openstack neutron/ml2. URL: https://wiki.openstack.org/wiki/Neutron/ML2.

48

Openstack cinder driver support matrix. URL: https://docs.openstack.org/cinder/latest/reference/support-matrix.html.

49

Openstack (cinder) available drivers. URL: https://docs.openstack.org/cinder/latest/drivers.html.

50

Openstack cinder service configuration. URL: https://docs.openstack.org/cinder/latest/configuration/index.html.

51

Openstack cinder administration. URL: https://docs.openstack.org/cinder/latest/admin/index.html.

52

Ceph - the future of storage. URL: https://ceph.io/en.

53

Keystone, the openstack identity service. URL: https://docs.openstack.org/keystone/wallaby/.

54

Openstack - welcome to glance's documentation! URL: https://docs.openstack.org/glance/wallaby/.

55

Openstack block storage (cinder) documentation. URL: https://docs.openstack.org/cinder/wallaby/.

56

Openstack - welcome to swift's documentation! URL: https://docs.openstack.org/swift/wallaby/.

57

Openstack - welcome to neutron's documentation! URL: https://docs.openstack.org/neutron/wallaby/.

58

Openstack - scenario: high availability using distributed virtual routing (dvr). URL: https://docs.openstack.org/liberty/networking-guide/scenario-dvr-ovs.html.

59

Openstack neutron: distributed virtual routing with vrrp. URL: https://docs.openstack.org/neutron/wallaby/admin/config-dvr-ha-snat.html.

60

Openstack compute (nova). URL: https://docs.openstack.org/nova/wallaby/.

61

Openstack - welcome to ironic's documentation! URL: https://docs.openstack.org/ironic/wallaby/.

62

Openstack ironic api reference: bare metal api. URL: https://docs.openstack.org/api-ref/baremetal/.

63

Openstack - welcome to the heat documentation! URL: https://docs.openstack.org/heat/wallaby/.

64

Horizon: the openstack dashboard project. URL: https://docs.openstack.org/horizon/wallaby/.

65

Openstack - placement. URL: https://docs.openstack.org/placement/wallaby/index.html.

66

Openstack - placement: modeling with provider trees. URL: https://docs.openstack.org/placement/latest/user/provider-tree.html.

67

Openstack - placement usage. URL: https://docs.openstack.org/placement/latest/user/index.html.

68

Openstack key manager (barbican). URL: https://docs.openstack.org/barbican/wallaby/.

69

Openstack accelerator (cyborg). URL: https://docs.openstack.org/cyborg/wallaby/.

70

Openstack compute api guide 2.1.0: server concepts. URL: https://docs.openstack.org/api-guide/compute/server_concepts.html.

71

Openstack cyborg support matrix (wallaby). URL: https://docs.openstack.org/cyborg/wallaby/reference/support-matrix.html.

72

Openstack cyborg support matrix. URL: https://docs.openstack.org/cyborg/latest/reference/support-matrix.html.

73

Openstack cyborg architecture. URL: https://docs.openstack.org/cyborg/latest/user/architecture.html.

74

Openstack nova: flavors. URL: https://docs.openstack.org/nova/latest/user/flavors.html.

75

Open glossary of edge computing. URL: https://github.com/State-of-the-Edge/glossary/blob/master/edge-glossary.md.

76

Openstack - edge computing: next steps in architecture, design and testing. URL: https://www.openstack.org/use-cases/edge-computing/edge-computing-next-steps-in-architecture-design-and-testing.

77

Openstack reference architecture for 100, 300 and 500 nodes: services placement summary. URL: https://fuel-ccp.readthedocs.io/en/latest/design/ref_arch_100_nodes.html#services-placement-summary.

78

Openstack nova: image pre-caching. URL: https://docs.openstack.org/nova/latest/admin/image-caching.html#image-pre-caching.

79

Airship v2. URL: https://www.airshipit.org/.

80

Starlingx - deploy your edge cloud now. URL: https://www.starlingx.io/.

81

Openstack tripleo. URL: http://opendev.org/openstack/tripleo-common.

82

Openstack compute microversions. URL: https://docs.openstack.org/api-guide/compute/microversions.html.

83

Identity api v3. URL: https://docs.openstack.org/api-ref/identity/v3/index.html.

84

Identity api v3 extensions. URL: https://docs.openstack.org/api-ref/identity/v3-ext/.

85

Security compliance and pci-dss. URL: https://docs.openstack.org/keystone/wallaby/admin/configuration.html#security-compliance-and-pci-dss.

86

Image service api. URL: https://docs.openstack.org/api-ref/image/v2/.

87

Image service versions. URL: https://docs.openstack.org/api-ref/image/versions/index.html#version-history.

88

Block storage api. URL: https://docs.openstack.org/api-ref/block-storage/.

89

Cinder rest api version history. URL: https://docs.openstack.org/cinder/latest/contributor/api_microversion_history.html.

90

Object storage api. URL: https://docs.openstack.org/api-ref/object-store/index.html.

91

Discoverability. URL: https://docs.openstack.org/swift/latest/api/discoverability.html.

92

Networking service apis. URL: https://docs.openstack.org/api-ref/network/.

93

Networking api v2.0. URL: https://docs.openstack.org/api-ref/network/v2/.

94

Compute api. URL: https://docs.openstack.org/api-ref/compute/.

95

Compute rest api version history. URL: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html.

96

Placement api. URL: https://docs.openstack.org/api-ref/placement/.

97

Placement rest api version history. URL: https://docs.openstack.org/placement/latest/placement-api-microversion-history.html.

98

Orchestration service api. URL: https://docs.openstack.org/api-ref/orchestration/.

99

Template version history. URL: https://docs.openstack.org/heat/latest/template_guide/hot_spec.html.

100

Heat orchestration template (hot) specification. URL: https://docs.openstack.org/heat/latest/template_guide/hot_spec.html#rocky.

101

Openstack apis. URL: https://docs.openstack.org/api-ref/.

102

Kubernetes apis. URL: https://kubernetes.io/docs/concepts/overview/kubernetes-api/.

103

Kvm apis. URL: https://www.kernel.org/doc/Documentation/virtual/kvm/api.txt.

104

Libvirt apis. URL: https://libvirt.org/html/index.html.

105

Barbican api. URL: https://docs.openstack.org/barbican/latest/api/.

106

Openstack - security boundaries and threats. URL: https://docs.openstack.org/security-guide/introduction/security-boundaries-and-threats.html.

107

Openstack security guide. URL: https://docs.openstack.org/security-guide/introduction/introduction-to-openstack.html.

108

Mitre - common vulnerabilities and exposures. URL: https://cve.mitre.org/.

109

National institute of standards and technology vulnerabilities metrics. URL: https://nvd.nist.gov/vuln-metrics/cvss.

110

Openstack security guide - identity. URL: https://docs.openstack.org/security-guide/identity.html.

111

Openstack security guide - authentication methods. URL: https://docs.openstack.org/security-guide/identity/authentication-methods.html.

112

Openstack security guide - policies. URL: https://docs.openstack.org/security-guide/identity/policies.html#policy-section.

113

Openstack keystone default roles. URL: https://docs.openstack.org/keystone/latest/admin/service-api-protection.html.

114

Openstack - introduction to tls and ssl. URL: https://docs.openstack.org/security-guide/secure-communication/introduction-to-ssl-and-tls.html.

115

Center for internet security cis-cat pro. URL: https://www.cisecurity.org/cybersecurity-tools/cis-cat-pro/.

116

Center for internet security benchmarks. URL: https://www.cisecurity.org/cis-benchmarks/.

117

Openstack image signature verification. URL: https://docs.openstack.org/glance/wallaby/user/signature.html.

118

Openstack - sr-iov passthrough for networking. URL: https://wiki.openstack.org/wiki/SR-IOV-Passthrough-For-Networking.

119

Openstack trusted images. URL: https://docs.openstack.org/security-guide/instance-management/security-services-for-instances.html#trusted-images.

120

Adding signed images. URL: https://docs.openstack.org/operations-guide/ops-user-facing-operations.html#adding-signed-images.

121

Network functions virtualisation (nfv) release 4; protocols and data models; vnf package and pnfd archive specification. ETSI GS NFV-SOL 004 V4.3.1, July 2022. URL: https://www.etsi.org/deliver/etsi_gs/NFV-SOL/001_099/004/04.03.01_60/gs_NFV-SOL004v040301p.pdf.

122

Network functions virtualisation (nfv) release 2; security; vnf package security specification. ETSI GS NFV-SEC 021 V2.6.1, June 2019. URL: https://www.etsi.org/deliver/etsi_gs/NFV-SEC/001_099/021/02.06.01_60/gs_nfv-sec021v020601p.pdf.

123

Foreman. URL: https://www.theforeman.org/.

124

Ansible documentation. URL: https://docs.ansible.com/.

125

Openstack tripleo architecture. URL: https://docs.openstack.org/tripleo-docs/latest/install/introduction/architecture.html#project-architecture.

126

Ovp. URL: https://www.opnfv.org/verification.

127

Jenkins. URL: https://build.opnfv.org/.

128

Test dabase. URL: https://docs.opnfv.org/en/stable-hunter/_images/OPNFV_testing_working_group.png.

129

S3 compatible storage service. URL: http://artifacts.opnfv.org/.

130

Functest-wallaby-zip. URL: https://build.opnfv.org/ci/job/functest-wallaby-zip/4/console.

131

Xtesting ci. URL: https://galaxy.ansible.com/collivier/xtesting.

132

Docker. URL: https://www.docker.com/.

133

Xtesting. URL: https://xtesting.readthedocs.io/en/latest/.

134

Opnfv fraser. URL: https://www.sdxcentral.com/articles/news/opnfvs-6th-release-brings-testing-capabilities-that-orange-is-already-using/2018/05/.

135

Xtesting python package. URL: https://pypi.org/project/xtesting/.

136

Test case execution description. URL: https://git.opnfv.org/functest-xtesting/tree/docker/core/testcases.yaml.

137

Ci/cd toolchains in a few commands. URL: https://github.com/collivier/ansible-role-xtesting#readme.

138

Ci/cd deployment models. URL: https://lists.opnfv.org/g/opnfv-tsc/message/5702.

139

Anuket releng. URL: https://git.opnfv.org/releng/tree/jjb/functest.

140

Test case result dump. URL: http://artifacts.opnfv.org/functest/9ID39XK47PMZ.zip.

141

Xtesting samples. URL: https://git.opnfv.org/functest-xtesting/plain/ansible/site.yml?h=stable/wallaby.

142

Openstack verification. URL: https://git.opnfv.org/functest/plain/ansible/site.yml?h=stable/wallaby.

143

Anuket rc1. URL: https://git.opnfv.org/functest/plain/ansible/site.cntt.yml?h=stable/wallaby.

144

Kubernetes verification. URL: https://git.opnfv.org/functest-kubernetes/plain/ansible/site.yml?h=stable/v1.22.

145

Functest. URL: https://functest.readthedocs.io/en/stable-wallaby/.

146

Refstack. URL: https://refstack.openstack.org/.

147

Networking bgpvpn. URL: https://docs.openstack.org/networking-bgpvpn/latest/.

148

Networking sfc. URL: https://docs.openstack.org/networking-sfc/latest/.

149

Devstack gates. URL: https://docs.opendev.org/opendev/system-config/latest/devstack-gate.html.

150

Rally. URL: https://github.com/openstack/rally-openstack.

151

Temptest. URL: https://github.com/openstack/tempest.

152

Temptest. URL: https://docs.openstack.org/devstack/latest/.

153

Raspberry pi. URL: https://www.raspberrypi.org/.

154

Functest daily jobs. URL: https://build.opnfv.org/ci/view/functest/job/functest-wallaby-daily/17/.

155

Openstack performance tools. URL: https://docs.openstack.org/developer/performance-docs/methodologies/tools.html.

156

Run alpine functest containers (wallaby). URL: https://wiki.anuket.io/display/HOME/Functest+Wallaby.

157

Functest gates. URL: https://build.opnfv.org/ci/view/functest.

158

New functest cntt containers. URL: https://lists.opnfv.org/g/opnfv-tsc/message/5717.

159

Keystone-tempest-plugin. URL: https://opendev.org/openstack/keystone-tempest-plugin.

160

Functest smoke cntt. URL: https://git.opnfv.org/functest/tree/docker/smoke-cntt/testcases.yaml?h=stable%2Fwallaby.

161

Cinder-tempest-plugin. URL: https://opendev.org/openstack/cinder-tempest-plugin.

162

Functest review 68881. URL: https://gerrit.opnfv.org/gerrit/68881.

163

Functest review 71011. URL: https://gerrit.opnfv.org/gerrit/71011.

164

Openstack bug 1770179. URL: https://launchpad.net/bugs/1770179.

165

Openstack bug 1677525. URL: https://launchpad.net/bugs/1677525.

166

Openstack bug 1317133. URL: https://launchpad.net/bugs/1317133.

167

Openstack bug 1905432. URL: https://launchpad.net/bugs/1905432.

168

Neutron-tempest-plugin. URL: https://opendev.org/openstack/neutron-tempest-plugin.

169

Openstack bug 1863707. URL: https://launchpad.net/bugs/1863707.

170

Functest review 69105. URL: https://gerrit.opnfv.org/gerrit/69105.

171

Openstack bug 1676207. URL: https://launchpad.net/bugs/1676207.

172

Openstack bug 1836595. URL: https://launchpad.net/bugs/1836595.

173

Openstack bug 1186354. URL: https://launchpad.net/bugs/1186354.

174

Openstack bug 1014647. URL: https://launchpad.net/bugs/1014647.

175

Openstack bug 1311500. URL: https://launchpad.net/bugs/1311500.

176

Openstack bug 1161411. URL: https://launchpad.net/bugs/1161411.

177

Openstack bug 1540645. URL: https://launchpad.net/bugs/1540645.

178

Heat-tempest-plugin. URL: https://opendev.org/openstack/heat-tempest-plugin.

179

Openstack story 2007804. URL: https://storyboard.openstack.org/#!/story/2007804.

180

Functest review 69926. URL: https://gerrit.opnfv.org/gerrit/69926.

181

Functest review 69931. URL: https://gerrit.opnfv.org/gerrit/69931.

182

Functest review 70004. URL: https://gerrit.opnfv.org/gerrit/70004.

183

Tempest-horizon. URL: https://github.com/openstack/tempest-horizon.

184

Functest healthcheck. URL: https://git.opnfv.org/functest/tree/docker/healthcheck/testcases.yaml?h=stable%2Fwallaby.

185

Functest benchmarking cntt. URL: https://git.opnfv.org/functest/tree/docker/benchmarking-cntt/testcases.yaml?h=stable%2Fwallaby.

186

Rally_full_cntt. URL: http://artifacts.opnfv.org/functest/KDBNITEN317M/functest-opnfv-functest-benchmarking-cntt-wallaby-rally_full_cntt-run-5/rally_full_cntt/rally_full_cntt.html.

187

Rally_jobs_cntt. URL: http://artifacts.opnfv.org/functest/KDBNITEN317M/functest-opnfv-functest-benchmarking-cntt-wallaby-rally_jobs_cntt-run-5/rally_jobs_cntt/rally_jobs_cntt.html.

188

Vmtp. URL: http://vmtp.readthedocs.io/en/latest.

189

Shaker. URL: https://pyshaker.readthedocs.io/en/latest/.

190

Vmtp scenarios. URL: http://artifacts.opnfv.org/functest/KDBNITEN317M/functest-opnfv-functest-benchmarking-wallaby-vmtp-run-8/vmtp/vmtp.json.

191

Functest vmtp. URL: http://artifacts.opnfv.org/functest/KDBNITEN317M/functest-opnfv-functest-benchmarking-wallaby-vmtp-run-8/vmtp/vmtp.html.

192

Shaker scenarios. URL: http://artifacts.opnfv.org/functest/KDBNITEN317M/functest-opnfv-functest-benchmarking-wallaby-shaker-run-8/shaker/report.json.

193

Functest vnf. URL: https://git.opnfv.org/functest/tree/docker/vnf/testcases.yaml?h=stable%2Fwallaby.

194

Clearwater ims. URL: https://clearwater.readthedocs.io/en/stable/.

195

Vyos vrouter. URL: https://www.vyos.io/.

196

Openairinterface vepc. URL: https://www.openairinterface.org/.

197

Cloudify. URL: https://cloudify.co.

198

Juju. URL: https://jaas.ai/.

199

Clearwater-live-test. URL: https://github.com/Metaswitch/clearwater-live-test.

200

Docker http/https proxy. URL: https://docs.docker.com/config/daemon/systemd/#httphttps-proxy.

201

Openstack autoscaling with heat. URL: https://docs.openstack.org/senlin/latest/scenarios/autoscaling_heat.html.