Kvm4nfv Configuration Guide

Danube 1.0

Configuration Abstract

This document provides guidance for the configurations available in the Danube release of OPNFV

The release includes four installer tools leveraging different technologies; Apex, Compass4nfv, Fuel and JOID, which deploy components of the platform.

This document also includes the selection of tools and components including guidelines for how to deploy and configure the platform to an operational state.

Configuration Options

OPNFV provides a variety of virtual infrastructure deployments called scenarios designed to host virtualised network functions (VNF’s). KVM4NFV scenarios provide specific capabilities and/or components aimed to solve specific problems for the deployment of VNF’s. KVM4NFV scenario includes components such as OpenStack,KVM etc. which includes different source components or configurations.

Note

  • Each KVM4NFV scenario provides unique features and capabilities, it is important to understand your target platform capabilities before installing and configuring. This configuration guide outlines how to configure components in order to enable the features required.
  • More deatils of kvm4nfv scenarios installation and description can be found in the scenario guide of kvm4nfv docs

Low Latency Feature Configuration Description

Introduction

In KVM4NFV project, we focus on the KVM hypervisor to enhance it for NFV, by looking at the following areas initially

  • Minimal Interrupt latency variation for data plane VNFs:
    • Minimal Timing Variation for Timing correctness of real-time VNFs
    • Minimal packet latency variation for data-plane VNFs
  • Inter-VM communication,
  • Fast live migration

Configuration of Cyclictest

Cyclictest measures Latency of response to a stimulus. Achieving low latency with the KVM4NFV project requires setting up a special test environment. This environment includes the BIOS settings, kernel configuration, kernel parameters and the run-time environment.

Pre-configuration activities

Intel POD10 is currently used as OPNFV-KVM4NFV test environment. The rpm packages from the latest build are downloaded onto Intel-Pod10 jump server from artifact repository. Yardstick running in a ubuntu docker container on Intel Pod10-jump server will configure the host(intel pod10 node1/node2 based on job type), the guest and triggers the cyclictest on the guest using below sample yaml file.

For IDLE-IDLE test,

host_setup_seqs:
- "host-setup0.sh"
- "reboot"
- "host-setup1.sh"
- "host-run-qemu.sh"

guest_setup_seqs:
- "guest-setup0.sh"
- "reboot"
- "guest-setup1.sh"
_images/idle-idle-test.png 
For [CPU/Memory/IO]Stress-IDLE tests,

host_setup_seqs:
- "host-setup0.sh"
- "reboot"
- "host-setup1.sh"
- "stress_daily.sh" [cpustress/memory/io]
- "host-run-qemu.sh"

guest_setup_seqs:
- "guest-setup0.sh"
- "reboot"
- "guest-setup1.sh"
_images/stress-idle-test.png

The following scripts are used for configuring host and guest to create a special test environment and achieve low latency.

Note: host-setup0.sh, host-setup1.sh and host-run-qemu.sh are run on the host, followed by guest-setup0.sh and guest-setup1.sh scripts on the guest VM.

host-setup0.sh: Running this script will install the latest kernel rpm on host and will make necessary changes as following to create special test environment.

  • Isolates CPUs from the general scheduler
  • Stops timer ticks on isolated CPUs whenever possible
  • Stops RCU callbacks on isolated CPUs
  • Enables intel iommu driver and disables DMA translation for devices
  • Sets HugeTLB pages to 1GB
  • Disables machine check
  • Disables clocksource verification at runtime

host-setup1.sh: Running this script will make the following test environment changes.

  • Disabling watchdogs to reduce overhead
  • Disabling RT throttling
  • Reroute interrupts bound to isolated CPUs to CPU 0
  • Change the iptable so that we can ssh to the guest remotely

stress_daily.sh: Scripts gets triggered only for stress-idle tests. Running this script make the following environment changes.

  • Triggers stress_script.sh, which runs the stress command with necessary options
  • CPU,Memory or IO stress can be applied based on the test type
  • Applying stress only on the Host is handled in D-Release
  • For Idle-Idle test the stress script is not triggered
  • Stress is applied only on the free cores to prevent load on qemu process
Note:
  • On Numa Node 1: 22,23 cores are allocated for QEMU process
  • 24-43 are used for applying stress
host-run-qemu.sh: Running this script will launch a guest vm on the host.
Note: download guest disk image from artifactory.

guest-setup0.sh: Running this scrcipt on the guest vm will install the latest build kernel rpm, cyclictest and make the following configuration on guest vm.

  • Isolates CPUs from the general scheduler
  • Stops timer ticks on isolated CPUs whenever possible
  • Uses polling idle loop to improve performance
  • Disables clocksource verification at runtime

guest-setup1.sh: Running this script on guest vm will do the following configurations.

  • Disable watchdogs to reduce overhead
  • Routes device interrupts to non-RT CPU
  • Disables RT throttling
Hardware configuration

Currently Intel POD10 is used as test environment for kvm4nfv to execute cyclictest. As part of this test environment Intel pod10-jump is configured as jenkins slave and all the latest build artifacts are downloaded on to it.