VSPERF Results

1. VSPERF Test Scenarios

Predefined Tests suitable for automated execution with CI:

Test Definition
phy2phy_tput PacketLossRatio for Phy2Phy
back2back BackToBackFrames for Phy2Phy
phy2phy_tput_mod_vlan PacketLossRatioFrameModification for Phy2Phy
phy2phy_cont Phy2Phy blast vswitch at x% TX rate and measure throughput
pvp_cont PVP blast vswitch at x% TX rate and measure throughput
pvvp_cont PVVP blast vswitch at x% TX rate and measure throughput
phy2phy_scalability Scalability0PacketLoss for Phy2Phy
pvp_tput PacketLossRatio for PVP
pvp_back2back BackToBackFrames for PVP
pvvp_tput PacketLossRatio for PVVP
pvvp_back2back BackToBackFrames for PVVP
phy2phy_cpu_load CPU0PacketLoss for Phy2Phy
phy2phy_mem_load Same as CPU0PacketLoss but using a memory intensive app

Deployment topologies:

  • Phy2Phy: Physical port -> vSwitch -> Physical port.
  • PVP: Physical port -> vSwitch -> VNF -> vSwitch -> Physical port.
  • PVVP: Physical port -> vSwitch -> VNF -> vSwitch -> VNF -> vSwitch -> Physical port.

Loopback applications in the Guest:

Supported traffic generators:

  • Spirent Testcenter
  • Ixia: IxOS and IxNet.
  • Xena
  • MoonGen
  • Dummy
  • T-Rex

2. OPNFV Test Results

VSPERF CI jobs are run daily and sample results can be found at https://wiki.opnfv.org/display/vsperf/Vsperf+Results

Testcase names shown in the dashboard are combination of orignal testcase name from VSPERF framework and indication of used vswitch.

Example:

Testcase phy2phy_tput is executed for three vSwitch types: OvsDpdkVhost, OvsVanilla and VppDpdkVhost. In this case, following testcase names will be used in the dashboard: phy2phy_tput_ovsdpdkvhost, phy2phy_tput_ovsvanilla and phy2phy_tput_vppdpdkvhost.

In case of RFC2544 Throughput test, the recorded metric is FPS (frames per second) without packet loss. For RFC2544 Back2Back test, the recorded metric is back-to-back value (number of frames) without packet loss.

The loopback application in the VNF used for PVP and PVVP scenarios was DPDK testpmd.

Guest interface types are vhost-user for OvsDpdkVhost and VppDpdkVhost and virtio-net for OvsVanilla.

3. Performance report for OvsDpdkVhost

3.1. Introduction

The objective of the OPNFV project titled “Characterise vSwitch Performance for Telco NFV Use Cases”, is to evaluate a virtual switch to identify its suitability for a Telco Network Function Virtualization (NFV) environment. As well as this, the project aims to identify any gaps or bottlenecks in order to drive architectural changes to improve virtual switch performance and determinism. The purpose of this document is to summarize the results of the tests carried out on the virtual switch in the Network Function Virtualization Infrastructure (NFVI) and, from these results, provide evaluations and recommendations for the virtual switch. Test results will be outlined in details-of-LTR, preceded by the document-identifier and the scope and references).

This document is currently in draft form.

3.1.1. Document identifier

The document id will be used to uniquely identify versions of the LTR. The format for the document id will be: OPNFV_vswitchperf_LTR_rel_STATUS, the status is one of: DRAFT, REVIEWED, CORRECTED or FINAL. The document id for this version of the LTR is: OPNFV_vswitchperf_LTR_Brahmaputra_DRAFT.

3.1.2. Scope

The scope of this report is to detail the results of the tests that have been performed on the virtual switch. This report will also evaluate the results of these tests and, based on these evaluations, provide recommendations on the suitability of the virtual switch for use in a Telco NFV environment.

3.2. Details of the Level Test Report

This section provides a test-results-overview. Also included are the rationale and the conclusions.

3.2.1. Test ID: BACK2BACK

3.2.1.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: c79adfe7660ffa43f597af794412c0616a785943

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: p2p
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
3.2.1.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.1.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 20861
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 64
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:31:01
start_time 2018-05-29 03:22:46
stop_time 2018-05-29 03:53:48
3.2.1.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.15
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056816
RSS 43063
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50700
RSS 6708
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.1.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 253378378
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 128
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:31:01
start_time 2018-05-29 03:22:46
stop_time 2018-05-29 03:53:48
3.2.1.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.15
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056816
RSS 43063
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50700
RSS 6708
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.1.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 70488721
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:31:01
start_time 2018-05-29 03:22:46
stop_time 2018-05-29 03:53:48
3.2.1.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.15
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056816
RSS 43063
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50700
RSS 6708
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.1.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 35919540
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:31:01
start_time 2018-05-29 03:22:46
stop_time 2018-05-29 03:53:48
3.2.1.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.15
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056816
RSS 43063
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50700
RSS 6708
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.1.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 24382314
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:31:01
start_time 2018-05-29 03:22:46
stop_time 2018-05-29 03:53:48
3.2.1.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.15
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056816
RSS 43063
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50700
RSS 6708
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.7. Anomalies

No anomalies were detected during the course of this test.

3.2.1.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.2. Test ID: PHY2PHY_SCALABILITY

3.2.2.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: c79adfe7660ffa43f597af794412c0616a785943

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: phy2phy_scalability
  • Description: LTD.Scalability.Flows.RFC2544.0PacketLoss
  • Deployment: p2p
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
3.2.2.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.2.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 15490171.219
tx_rate_mbps Unknown
throughput_rx_mbps 7930.968
tx_rate_percent 52.047
throughput_rx_percent 52.047
frame_loss_percent 0.000
min_latency_ns 5120.000
max_latency_ns 94880.000
avg_latency_ns 14385.500
type rfc2544_throughput
packet_size 64
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:17:42
start_time 2018-05-29 04:21:39
stop_time 2018-05-29 04:39:22
stream_count 8000
match_type L4
pre-installed_flows No
3.2.2.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.13
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 5056816
RSS 43138
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50712
RSS 6856
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.2.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.2.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 15585402.265
tx_rate_mbps Unknown
throughput_rx_mbps 15959.452
tx_rate_percent 92.266
throughput_rx_percent 92.266
frame_loss_percent 0.000
min_latency_ns 5440.000
max_latency_ns 141560.000
avg_latency_ns 29980.500
type rfc2544_throughput
packet_size 128
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:17:42
start_time 2018-05-29 04:21:39
stop_time 2018-05-29 04:39:22
stream_count 8000
match_type L4
pre-installed_flows No
3.2.2.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.13
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 5056816
RSS 43138
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50712
RSS 6856
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.2.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.2.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699200.420
tx_rate_mbps Unknown
throughput_rx_mbps 19247.925
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 15760.000
max_latency_ns 49320.000
avg_latency_ns 36662.000
type rfc2544_throughput
packet_size 512
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:17:42
start_time 2018-05-29 04:21:39
stop_time 2018-05-29 04:39:22
stream_count 8000
match_type L4
pre-installed_flows No
3.2.2.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.13
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 5056816
RSS 43138
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50712
RSS 6856
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.2.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.2.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394615.059
tx_rate_mbps Unknown
throughput_rx_mbps 19616.687
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 16380.000
max_latency_ns 46300.000
avg_latency_ns 32790.000
type rfc2544_throughput
packet_size 1024
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:17:42
start_time 2018-05-29 04:21:39
stop_time 2018-05-29 04:39:22
stream_count 8000
match_type L4
pre-installed_flows No
3.2.2.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.13
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 5056816
RSS 43138
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50712
RSS 6856
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.2.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.2.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625474.557
tx_rate_mbps Unknown
throughput_rx_mbps 19739.763
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 14780.000
max_latency_ns 44800.000
avg_latency_ns 30558.500
type rfc2544_throughput
packet_size 1518
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:17:42
start_time 2018-05-29 04:21:39
stop_time 2018-05-29 04:39:22
stream_count 8000
match_type L4
pre-installed_flows No
3.2.2.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.13
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 5056816
RSS 43138
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50712
RSS 6856
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.2.7. Anomalies

No anomalies were detected during the course of this test.

3.2.2.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.3. Test ID: PHY2PHY_TPUT_MOD_VLAN

3.2.3.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: c79adfe7660ffa43f597af794412c0616a785943

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: phy2phy_tput_mod_vlan
  • Description: LTD.Throughput.RFC2544.PacketLossRatioFrameModification
  • Deployment: p2p
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
3.2.3.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.3.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 14569459.021
tx_rate_mbps Unknown
throughput_rx_mbps 7925.786
tx_rate_percent 48.953
throughput_rx_percent 51.284
frame_loss_percent 0.000
min_latency_ns 5180.000
max_latency_ns 29580.000
avg_latency_ns 11924.000
type rfc2544_throughput
packet_size 64
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:27:50
start_time 2018-05-29 03:53:49
stop_time 2018-05-29 04:21:39
3.2.3.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.15
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056812
RSS 42627
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6856
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.3.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.3.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 14670903.626
tx_rate_mbps Unknown
throughput_rx_mbps 15492.474
tx_rate_percent 86.852
throughput_rx_percent 89.199
frame_loss_percent 0.000
min_latency_ns 5600.000
max_latency_ns 88880.000
avg_latency_ns 15521.500
type rfc2544_throughput
packet_size 128
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:27:50
start_time 2018-05-29 03:53:49
stop_time 2018-05-29 04:21:39
3.2.3.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.15
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056812
RSS 42627
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6856
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.3.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.3.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4662904.166
tx_rate_mbps Unknown
throughput_rx_mbps 19248.468
tx_rate_percent 99.227
throughput_rx_percent 99.973
frame_loss_percent 0.000
min_latency_ns 5960.000
max_latency_ns 23820.000
avg_latency_ns 10878.500
type rfc2544_throughput
packet_size 512
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:27:50
start_time 2018-05-29 03:53:49
stop_time 2018-05-29 04:21:39
3.2.3.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.15
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056812
RSS 42627
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6856
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.3.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.3.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2376116.826
tx_rate_mbps Unknown
throughput_rx_mbps 19541.185
tx_rate_percent 99.227
throughput_rx_percent 99.607
frame_loss_percent 0.000
min_latency_ns 6600.000
max_latency_ns 22660.000
avg_latency_ns 9395.000
type rfc2544_throughput
packet_size 1024
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:27:50
start_time 2018-05-29 03:53:49
stop_time 2018-05-29 04:21:39
3.2.3.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.15
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056812
RSS 42627
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6856
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.3.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.3.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1612916.347
tx_rate_mbps Unknown
throughput_rx_mbps 19638.869
tx_rate_percent 99.227
throughput_rx_percent 99.485
frame_loss_percent 0.000
min_latency_ns 6920.000
max_latency_ns 24080.000
avg_latency_ns 9555.000
type rfc2544_throughput
packet_size 1518
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:27:50
start_time 2018-05-29 03:53:49
stop_time 2018-05-29 04:21:39
3.2.3.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.15
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056812
RSS 42627
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6856
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.3.7. Anomalies

No anomalies were detected during the course of this test.

3.2.3.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.4. Test ID: PVP_BACK2BACK

3.2.4.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: c79adfe7660ffa43f597af794412c0616a785943

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

  • VNF: QemuDpdkVhostUser, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-16.04_trex_20180209.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvp_back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: pvp
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
3.2.4.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.4.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 4257
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 64
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:50:33
start_time 2018-05-29 04:58:22
stop_time 2018-05-29 05:48:56
guest_loopback_app testpmd
3.2.4.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.16
%system 0.02
%guest 0.00
%CPU 400.18
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 11422004
RSS 43096
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50702
RSS 6767
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.06
%system 0.06
%guest 200.04
%CPU 200.17
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 2816829
RSS 70885
%MEM 0.11
kB_rd/s 0.00
kB_wr/s 7.37
kB_ccwr/s 0.00
3.2.4.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.4.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1449
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 128
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:50:33
start_time 2018-05-29 04:58:22
stop_time 2018-05-29 05:48:56
guest_loopback_app testpmd
3.2.4.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.16
%system 0.02
%guest 0.00
%CPU 400.18
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 11422004
RSS 43096
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50702
RSS 6767
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.06
%system 0.06
%guest 200.04
%CPU 200.17
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 2816829
RSS 70885
%MEM 0.11
kB_rd/s 0.00
kB_wr/s 7.37
kB_ccwr/s 0.00
3.2.4.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.4.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 70488721
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:50:33
start_time 2018-05-29 04:58:22
stop_time 2018-05-29 05:48:56
guest_loopback_app testpmd
3.2.4.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.16
%system 0.02
%guest 0.00
%CPU 400.18
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 11422004
RSS 43096
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50702
RSS 6767
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.06
%system 0.06
%guest 200.04
%CPU 200.17
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 2816829
RSS 70885
%MEM 0.11
kB_rd/s 0.00
kB_wr/s 7.37
kB_ccwr/s 0.00
3.2.4.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.4.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 35919540
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:50:33
start_time 2018-05-29 04:58:22
stop_time 2018-05-29 05:48:56
guest_loopback_app testpmd
3.2.4.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.16
%system 0.02
%guest 0.00
%CPU 400.18
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 11422004
RSS 43096
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50702
RSS 6767
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.06
%system 0.06
%guest 200.04
%CPU 200.17
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 2816829
RSS 70885
%MEM 0.11
kB_rd/s 0.00
kB_wr/s 7.37
kB_ccwr/s 0.00
3.2.4.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.4.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 24382314
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:50:33
start_time 2018-05-29 04:58:22
stop_time 2018-05-29 05:48:56
guest_loopback_app testpmd
3.2.4.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.16
%system 0.02
%guest 0.00
%CPU 400.18
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 11422004
RSS 43096
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50702
RSS 6767
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.06
%system 0.06
%guest 200.04
%CPU 200.17
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 2816829
RSS 70885
%MEM 0.11
kB_rd/s 0.00
kB_wr/s 7.37
kB_ccwr/s 0.00
3.2.4.7. Anomalies

No anomalies were detected during the course of this test.

3.2.4.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.5. Test ID: PHY2PHY_TPUT

3.2.5.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: c79adfe7660ffa43f597af794412c0616a785943

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: phy2phy_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: p2p
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
3.2.5.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.5.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 18943035.294
tx_rate_mbps Unknown
throughput_rx_mbps 9698.834
tx_rate_percent 63.648
throughput_rx_percent 63.648
frame_loss_percent 0.000
min_latency_ns 5640.000
max_latency_ns 84400.000
avg_latency_ns 10681.500
type rfc2544_throughput
packet_size 64
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:13:29
start_time 2018-05-29 03:09:16
stop_time 2018-05-29 03:22:46
3.2.5.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.15
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 5056812
RSS 42613
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6784
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.5.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.5.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 16891698.205
tx_rate_mbps Unknown
throughput_rx_mbps 17297.099
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 15840.000
max_latency_ns 55600.000
avg_latency_ns 41111.000
type rfc2544_throughput
packet_size 128
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:13:29
start_time 2018-05-29 03:09:16
stop_time 2018-05-29 03:22:46
3.2.5.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.15
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 5056812
RSS 42613
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6784
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.5.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.5.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699199.918
tx_rate_mbps Unknown
throughput_rx_mbps 19247.923
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 16520.000
max_latency_ns 47720.000
avg_latency_ns 37473.000
type rfc2544_throughput
packet_size 512
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:13:29
start_time 2018-05-29 03:09:16
stop_time 2018-05-29 03:22:46
3.2.5.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.15
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 5056812
RSS 42613
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6784
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.5.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.5.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394615.682
tx_rate_mbps Unknown
throughput_rx_mbps 19616.692
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 16420.000
max_latency_ns 43540.000
avg_latency_ns 32384.500
type rfc2544_throughput
packet_size 1024
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:13:29
start_time 2018-05-29 03:09:16
stop_time 2018-05-29 03:22:46
3.2.5.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.15
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 5056812
RSS 42613
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6784
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.5.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.5.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625474.579
tx_rate_mbps Unknown
throughput_rx_mbps 19739.763
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 16900.000
max_latency_ns 41720.000
avg_latency_ns 31195.000
type rfc2544_throughput
packet_size 1518
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:13:29
start_time 2018-05-29 03:09:16
stop_time 2018-05-29 03:22:46
3.2.5.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.15
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 5056812
RSS 42613
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6784
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.5.7. Anomalies

No anomalies were detected during the course of this test.

3.2.5.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.6. Test ID: PVP_TPUT

3.2.6.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: c79adfe7660ffa43f597af794412c0616a785943

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

  • VNF: QemuDpdkVhostUser, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-16.04_trex_20180209.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvp_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvp
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
3.2.6.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.6.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 7663713.809
tx_rate_mbps Unknown
throughput_rx_mbps 3923.821
tx_rate_percent 25.750
throughput_rx_percent 25.750
frame_loss_percent 0.000
min_latency_ns 9520.000
max_latency_ns 41280.000
avg_latency_ns 12989.000
type rfc2544_throughput
packet_size 64
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:18:59
start_time 2018-05-29 04:39:22
stop_time 2018-05-29 04:58:21
guest_loopback_app testpmd
3.2.6.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.16
%system 0.02
%guest 0.00
%CPU 400.18
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 11422000
RSS 43084
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6740
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.10
%system 0.08
%guest 200.06
%CPU 200.23
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 2916711
RSS 69408
%MEM 0.11
kB_rd/s 0.00
kB_wr/s 20.68
kB_ccwr/s 0.00
3.2.6.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.6.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 6440035.201
tx_rate_mbps Unknown
throughput_rx_mbps 6594.596
tx_rate_percent 38.125
throughput_rx_percent 38.125
frame_loss_percent 0.000
min_latency_ns 8900.000
max_latency_ns 39180.000
avg_latency_ns 12677.000
type rfc2544_throughput
packet_size 128
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:18:59
start_time 2018-05-29 04:39:22
stop_time 2018-05-29 04:58:21
guest_loopback_app testpmd
3.2.6.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.16
%system 0.02
%guest 0.00
%CPU 400.18
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 11422000
RSS 43084
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6740
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.10
%system 0.08
%guest 200.06
%CPU 200.23
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 2916711
RSS 69408
%MEM 0.11
kB_rd/s 0.00
kB_wr/s 20.68
kB_ccwr/s 0.00
3.2.6.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.6.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699181.952
tx_rate_mbps Unknown
throughput_rx_mbps 19247.849
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 26020.000
max_latency_ns 72240.000
avg_latency_ns 58641.500
type rfc2544_throughput
packet_size 512
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:18:59
start_time 2018-05-29 04:39:22
stop_time 2018-05-29 04:58:21
guest_loopback_app testpmd
3.2.6.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.16
%system 0.02
%guest 0.00
%CPU 400.18
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 11422000
RSS 43084
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6740
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.10
%system 0.08
%guest 200.06
%CPU 200.23
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 2916711
RSS 69408
%MEM 0.11
kB_rd/s 0.00
kB_wr/s 20.68
kB_ccwr/s 0.00
3.2.6.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.6.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394604.555
tx_rate_mbps Unknown
throughput_rx_mbps 19616.601
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 30340.000
max_latency_ns 71380.000
avg_latency_ns 58823.500
type rfc2544_throughput
packet_size 1024
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:18:59
start_time 2018-05-29 04:39:22
stop_time 2018-05-29 04:58:21
guest_loopback_app testpmd
3.2.6.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.16
%system 0.02
%guest 0.00
%CPU 400.18
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 11422000
RSS 43084
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6740
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.10
%system 0.08
%guest 200.06
%CPU 200.23
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 2916711
RSS 69408
%MEM 0.11
kB_rd/s 0.00
kB_wr/s 20.68
kB_ccwr/s 0.00
3.2.6.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.6.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625467.963
tx_rate_mbps Unknown
throughput_rx_mbps 19739.683
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 28560.000
max_latency_ns 66100.000
avg_latency_ns 55169.000
type rfc2544_throughput
packet_size 1518
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:18:59
start_time 2018-05-29 04:39:22
stop_time 2018-05-29 04:58:21
guest_loopback_app testpmd
3.2.6.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.16
%system 0.02
%guest 0.00
%CPU 400.18
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 11422000
RSS 43084
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6740
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.10
%system 0.08
%guest 200.06
%CPU 200.23
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 2916711
RSS 69408
%MEM 0.11
kB_rd/s 0.00
kB_wr/s 20.68
kB_ccwr/s 0.00
3.2.6.7. Anomalies

No anomalies were detected during the course of this test.

3.2.6.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.3. Rationale for decisions

The tests conducted do not have pass/fail/conditional-pass criteria. The test is simply conducted and the results are reported.

3.4. Conclusions and recommendations

The test results are stable. The vsperf CI jobs that were used to obtain the results can be found at https://artifacts.opnfv.org/logs/vswitchperf/intel-pod12/2018-05-29_10-09-09/vswitchperf_logs_2018-05-29_10-09-09.tar.gz.

3.5. General

3.5.1. Glossary

  • NFV - Network Function Virtualization
  • Mbps - 1,000,000bps

3.5.2. Document change procedures and history

Document ID Author Date Modified
OPNFV_vswitchperf_LTR_ver_1.0_Jan_15_CN_DRAFT Christopher Nolan 23/01/2015
OPNFV_vswitchperf_LTR_ver_1.1_Jan_15_CN_DRAFT Christopher Nolan 28/01/2015

4. Performance report for OvsVanilla

4.1. Introduction

The objective of the OPNFV project titled “Characterise vSwitch Performance for Telco NFV Use Cases”, is to evaluate a virtual switch to identify its suitability for a Telco Network Function Virtualization (NFV) environment. As well as this, the project aims to identify any gaps or bottlenecks in order to drive architectural changes to improve virtual switch performance and determinism. The purpose of this document is to summarize the results of the tests carried out on the virtual switch in the Network Function Virtualization Infrastructure (NFVI) and, from these results, provide evaluations and recommendations for the virtual switch. Test results will be outlined in details-of-LTR, preceded by the document-identifier and the scope and references).

This document is currently in draft form.

4.1.1. Document identifier

The document id will be used to uniquely identify versions of the LTR. The format for the document id will be: OPNFV_vswitchperf_LTR_rel_STATUS, the status is one of: DRAFT, REVIEWED, CORRECTED or FINAL. The document id for this version of the LTR is: OPNFV_vswitchperf_LTR_Brahmaputra_DRAFT.

4.1.2. Scope

The scope of this report is to detail the results of the tests that have been performed on the virtual switch. This report will also evaluate the results of these tests and, based on these evaluations, provide recommendations on the suitability of the virtual switch for use in a Telco NFV environment.

4.2. Details of the Level Test Report

This section provides a test-results-overview. Also included are the rationale and the conclusions.

4.2.1. Test ID: BACK2BACK

4.2.1.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: c79adfe7660ffa43f597af794412c0616a785943

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

Below are test details:

  • Test ID: back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: p2p
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
4.2.1.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.1.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 5108
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:42:18
start_time 2018-05-29 06:34:40
stop_time 2018-05-29 07:16:58
4.2.1.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.50
%system 1.05
%guest 0.00
%CPU 1.56
CPU
minflt/s 0.08
majflt/s 0.00
VSZ 3364104
RSS 7752
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44316
RSS 5816
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.1.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 3382
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:42:18
start_time 2018-05-29 06:34:40
stop_time 2018-05-29 07:16:58
4.2.1.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.50
%system 1.05
%guest 0.00
%CPU 1.56
CPU
minflt/s 0.08
majflt/s 0.00
VSZ 3364104
RSS 7752
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44316
RSS 5816
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.1.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1613
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:42:18
start_time 2018-05-29 06:34:40
stop_time 2018-05-29 07:16:58
4.2.1.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.50
%system 1.05
%guest 0.00
%CPU 1.56
CPU
minflt/s 0.08
majflt/s 0.00
VSZ 3364104
RSS 7752
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44316
RSS 5816
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.1.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1370
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:42:18
start_time 2018-05-29 06:34:40
stop_time 2018-05-29 07:16:58
4.2.1.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.50
%system 1.05
%guest 0.00
%CPU 1.56
CPU
minflt/s 0.08
majflt/s 0.00
VSZ 3364104
RSS 7752
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44316
RSS 5816
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.1.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1488
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:42:18
start_time 2018-05-29 06:34:40
stop_time 2018-05-29 07:16:58
4.2.1.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.50
%system 1.05
%guest 0.00
%CPU 1.56
CPU
minflt/s 0.08
majflt/s 0.00
VSZ 3364104
RSS 7752
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44316
RSS 5816
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.7. Anomalies

No anomalies were detected during the course of this test.

4.2.1.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.2. Test ID: PHY2PHY_SCALABILITY

4.2.2.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: c79adfe7660ffa43f597af794412c0616a785943

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

Below are test details:

  • Test ID: phy2phy_scalability
  • Description: LTD.Scalability.Flows.RFC2544.0PacketLoss
  • Deployment: p2p
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
4.2.2.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.2.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 757999.304
tx_rate_mbps Unknown
throughput_rx_mbps 388.096
tx_rate_percent 2.547
throughput_rx_percent 2.547
frame_loss_percent 0.000
min_latency_ns 4040.000
max_latency_ns 1154340.000
avg_latency_ns 28839.500
type rfc2544_throughput
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:44
start_time 2018-05-29 07:44:34
stop_time 2018-05-29 08:12:19
stream_count 8000
match_type L4
pre-installed_flows No
4.2.2.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.48
%system 1.16
%guest 0.00
%CPU 1.65
CPU
minflt/s 3.56
majflt/s 0.00
VSZ 3364112
RSS 16382
%MEM 0.02
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44316
RSS 6012
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.2.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.2.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 952809.255
tx_rate_mbps Unknown
throughput_rx_mbps 975.677
tx_rate_percent 5.641
throughput_rx_percent 5.641
frame_loss_percent 0.000
min_latency_ns 4200.000
max_latency_ns 4646880.000
avg_latency_ns 814579.000
type rfc2544_throughput
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:44
start_time 2018-05-29 07:44:34
stop_time 2018-05-29 08:12:19
stream_count 8000
match_type L4
pre-installed_flows No
4.2.2.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.48
%system 1.16
%guest 0.00
%CPU 1.65
CPU
minflt/s 3.56
majflt/s 0.00
VSZ 3364112
RSS 16382
%MEM 0.02
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44316
RSS 6012
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.2.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.2.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 810253.054
tx_rate_mbps Unknown
throughput_rx_mbps 3318.797
tx_rate_percent 17.242
throughput_rx_percent 17.242
frame_loss_percent 0.000
min_latency_ns 5040.000
max_latency_ns 393040.000
avg_latency_ns 29356.500
type rfc2544_throughput
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:44
start_time 2018-05-29 07:44:34
stop_time 2018-05-29 08:12:19
stream_count 8000
match_type L4
pre-installed_flows No
4.2.2.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.48
%system 1.16
%guest 0.00
%CPU 1.65
CPU
minflt/s 3.56
majflt/s 0.00
VSZ 3364112
RSS 16382
%MEM 0.02
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44316
RSS 6012
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.2.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.2.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 838871.630
tx_rate_mbps Unknown
throughput_rx_mbps 6872.036
tx_rate_percent 35.031
throughput_rx_percent 35.031
frame_loss_percent 0.000
min_latency_ns 5800.000
max_latency_ns 478760.000
avg_latency_ns 38684.500
type rfc2544_throughput
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:44
start_time 2018-05-29 07:44:34
stop_time 2018-05-29 08:12:19
stream_count 8000
match_type L4
pre-installed_flows No
4.2.2.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.48
%system 1.16
%guest 0.00
%CPU 1.65
CPU
minflt/s 3.56
majflt/s 0.00
VSZ 3364112
RSS 16382
%MEM 0.02
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44316
RSS 6012
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.2.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.2.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 833444.293
tx_rate_mbps Unknown
throughput_rx_mbps 10121.347
tx_rate_percent 51.273
throughput_rx_percent 51.273
frame_loss_percent 0.000
min_latency_ns 8660.000
max_latency_ns 450020.000
avg_latency_ns 39898.500
type rfc2544_throughput
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:44
start_time 2018-05-29 07:44:34
stop_time 2018-05-29 08:12:19
stream_count 8000
match_type L4
pre-installed_flows No
4.2.2.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.48
%system 1.16
%guest 0.00
%CPU 1.65
CPU
minflt/s 3.56
majflt/s 0.00
VSZ 3364112
RSS 16382
%MEM 0.02
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44316
RSS 6012
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.2.7. Anomalies

No anomalies were detected during the course of this test.

4.2.2.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.3. Test ID: PVVP_TPUT

4.2.3.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvvp

  • vswitchperf: GIT tag: c79adfe7660ffa43f597af794412c0616a785943

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • VNF: QemuVirtioNet, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-16.04_trex_20180209.qcow2
    • /home/jenkins/vloop-vnf-ubuntu-16.04_trex_20180209.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvvp_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvvp
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
4.2.3.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.3.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 242744.876
tx_rate_mbps Unknown
throughput_rx_mbps 124.285
tx_rate_percent 1.773
throughput_rx_percent 0.816
frame_loss_percent 54.008
min_latency_ns 34700.000
max_latency_ns 70293040.000
avg_latency_ns 4922360.000
type rfc2544_throughput
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:29:52
start_time 2018-05-29 09:24:17
stop_time 2018-05-29 09:54:09
guest_loopback_app testpmd testpmd
4.2.3.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.51
%system 0.96
%guest 0.00
%CPU 1.49
CPU
minflt/s 0.12
majflt/s 0.00
VSZ 3364236
RSS 8026
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5812
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 1.69
%guest 198.38
%CPU 200.09
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2833884
RSS 67815
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 12.62
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.06
%system 1.68
%guest 198.46
%CPU 200.19
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2840084
RSS 68356
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
4.2.3.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.3.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 299569.513
tx_rate_mbps Unknown
throughput_rx_mbps 306.759
tx_rate_percent 1.773
throughput_rx_percent 1.773
frame_loss_percent 0.000
min_latency_ns 21240.000
max_latency_ns 1271300.000
avg_latency_ns 72325.000
type rfc2544_throughput
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:29:52
start_time 2018-05-29 09:24:17
stop_time 2018-05-29 09:54:09
guest_loopback_app testpmd testpmd
4.2.3.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.51
%system 0.96
%guest 0.00
%CPU 1.49
CPU
minflt/s 0.12
majflt/s 0.00
VSZ 3364236
RSS 8026
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5812
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 1.69
%guest 198.38
%CPU 200.09
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2833884
RSS 67815
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 12.62
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.06
%system 1.68
%guest 198.46
%CPU 200.19
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2840084
RSS 68356
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
4.2.3.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.3.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 301413.842
tx_rate_mbps Unknown
throughput_rx_mbps 1234.591
tx_rate_percent 6.414
throughput_rx_percent 6.414
frame_loss_percent 0.000
min_latency_ns 29140.000
max_latency_ns 1311800.000
avg_latency_ns 116700.500
type rfc2544_throughput
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:29:52
start_time 2018-05-29 09:24:17
stop_time 2018-05-29 09:54:09
guest_loopback_app testpmd testpmd
4.2.3.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.51
%system 0.96
%guest 0.00
%CPU 1.49
CPU
minflt/s 0.12
majflt/s 0.00
VSZ 3364236
RSS 8026
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5812
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 1.69
%guest 198.38
%CPU 200.09
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2833884
RSS 67815
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 12.62
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.06
%system 1.68
%guest 198.46
%CPU 200.19
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2840084
RSS 68356
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
4.2.3.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.3.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 320298.491
tx_rate_mbps Unknown
throughput_rx_mbps 2623.885
tx_rate_percent 13.375
throughput_rx_percent 13.375
frame_loss_percent 0.000
min_latency_ns 38340.000
max_latency_ns 1065220.000
avg_latency_ns 134714.000
type rfc2544_throughput
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:29:52
start_time 2018-05-29 09:24:17
stop_time 2018-05-29 09:54:09
guest_loopback_app testpmd testpmd
4.2.3.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.51
%system 0.96
%guest 0.00
%CPU 1.49
CPU
minflt/s 0.12
majflt/s 0.00
VSZ 3364236
RSS 8026
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5812
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 1.69
%guest 198.38
%CPU 200.09
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2833884
RSS 67815
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 12.62
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.06
%system 1.68
%guest 198.46
%CPU 200.19
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2840084
RSS 68356
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
4.2.3.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.3.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 280270.522
tx_rate_mbps Unknown
throughput_rx_mbps 3403.605
tx_rate_percent 17.242
throughput_rx_percent 17.242
frame_loss_percent 0.000
min_latency_ns 28920.000
max_latency_ns 1287280.000
avg_latency_ns 107605.500
type rfc2544_throughput
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:29:52
start_time 2018-05-29 09:24:17
stop_time 2018-05-29 09:54:09
guest_loopback_app testpmd testpmd
4.2.3.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.51
%system 0.96
%guest 0.00
%CPU 1.49
CPU
minflt/s 0.12
majflt/s 0.00
VSZ 3364236
RSS 8026
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5812
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 1.69
%guest 198.38
%CPU 200.09
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2833884
RSS 67815
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 12.62
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.06
%system 1.68
%guest 198.46
%CPU 200.19
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2840084
RSS 68356
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
4.2.3.7. Anomalies

No anomalies were detected during the course of this test.

4.2.3.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.4. Test ID: PHY2PHY_TPUT_MOD_VLAN

4.2.4.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: c79adfe7660ffa43f597af794412c0616a785943

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

Below are test details:

  • Test ID: phy2phy_tput_mod_vlan
  • Description: LTD.Throughput.RFC2544.PacketLossRatioFrameModification
  • Deployment: p2p
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
4.2.4.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.4.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 757999.430
tx_rate_mbps Unknown
throughput_rx_mbps 412.352
tx_rate_percent 2.547
throughput_rx_percent 2.668
frame_loss_percent 0.000
min_latency_ns 4100.000
max_latency_ns 198180.000
avg_latency_ns 27947.000
type rfc2544_throughput
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:34
start_time 2018-05-29 07:16:59
stop_time 2018-05-29 07:44:34
4.2.4.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.49
%system 1.09
%guest 0.00
%CPU 1.59
CPU
minflt/s 0.17
majflt/s 0.00
VSZ 3364112
RSS 7758
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5816
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.4.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.4.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 822160.801
tx_rate_mbps Unknown
throughput_rx_mbps 868.202
tx_rate_percent 4.867
throughput_rx_percent 4.999
frame_loss_percent 0.000
min_latency_ns 4240.000
max_latency_ns 206060.000
avg_latency_ns 29393.500
type rfc2544_throughput
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:34
start_time 2018-05-29 07:16:59
stop_time 2018-05-29 07:44:34
4.2.4.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.49
%system 1.09
%guest 0.00
%CPU 1.59
CPU
minflt/s 0.17
majflt/s 0.00
VSZ 3364112
RSS 7758
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5816
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.4.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.4.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 773908.240
tx_rate_mbps Unknown
throughput_rx_mbps 3194.693
tx_rate_percent 16.469
throughput_rx_percent 16.593
frame_loss_percent 0.000
min_latency_ns 5080.000
max_latency_ns 424800.000
avg_latency_ns 29577.500
type rfc2544_throughput
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:34
start_time 2018-05-29 07:16:59
stop_time 2018-05-29 07:44:34
4.2.4.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.49
%system 1.09
%guest 0.00
%CPU 1.59
CPU
minflt/s 0.17
majflt/s 0.00
VSZ 3364112
RSS 7758
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5816
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.4.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.4.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 801829.617
tx_rate_mbps Unknown
throughput_rx_mbps 6594.247
tx_rate_percent 33.484
throughput_rx_percent 33.613
frame_loss_percent 0.000
min_latency_ns 5840.000
max_latency_ns 455980.000
avg_latency_ns 34402.500
type rfc2544_throughput
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:34
start_time 2018-05-29 07:16:59
stop_time 2018-05-29 07:44:34
4.2.4.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.49
%system 1.09
%guest 0.00
%CPU 1.59
CPU
minflt/s 0.17
majflt/s 0.00
VSZ 3364112
RSS 7758
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5816
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.4.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.4.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 808300.014
tx_rate_mbps Unknown
throughput_rx_mbps 9841.861
tx_rate_percent 49.727
throughput_rx_percent 49.856
frame_loss_percent 0.000
min_latency_ns 6720.000
max_latency_ns 764480.000
avg_latency_ns 42454.500
type rfc2544_throughput
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:34
start_time 2018-05-29 07:16:59
stop_time 2018-05-29 07:44:34
4.2.4.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.49
%system 1.09
%guest 0.00
%CPU 1.59
CPU
minflt/s 0.17
majflt/s 0.00
VSZ 3364112
RSS 7758
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5816
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.4.7. Anomalies

No anomalies were detected during the course of this test.

4.2.4.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.5. Test ID: PVVP_BACK2BACK

4.2.5.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvvp

  • vswitchperf: GIT tag: c79adfe7660ffa43f597af794412c0616a785943

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • VNF: QemuVirtioNet, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-16.04_trex_20180209.qcow2
    • /home/jenkins/vloop-vnf-ubuntu-16.04_trex_20180209.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvvp_back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: pvvp
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
4.2.5.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.5.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 425
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:44:26
start_time 2018-05-29 09:54:10
stop_time 2018-05-29 10:38:37
guest_loopback_app testpmd testpmd
4.2.5.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.51
%system 1.08
%guest 0.00
%CPU 1.56
CPU
minflt/s 0.15
majflt/s 0.00
VSZ 3364224
RSS 8133
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44316
RSS 5804
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.06
%system 0.17
%guest 199.95
%CPU 200.17
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2843168
RSS 67496
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.51
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.06
%system 0.16
%guest 199.96
%CPU 200.17
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2754788
RSS 63600
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.5.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.5.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544_back2back
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:44:26
start_time 2018-05-29 09:54:10
stop_time 2018-05-29 10:38:37
guest_loopback_app testpmd testpmd
4.2.5.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.51
%system 1.08
%guest 0.00
%CPU 1.56
CPU
minflt/s 0.15
majflt/s 0.00
VSZ 3364224
RSS 8133
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44316
RSS 5804
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.06
%system 0.17
%guest 199.95
%CPU 200.17
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2843168
RSS 67496
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.51
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.06
%system 0.16
%guest 199.96
%CPU 200.17
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2754788
RSS 63600
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.5.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.5.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 537
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:44:26
start_time 2018-05-29 09:54:10
stop_time 2018-05-29 10:38:37
guest_loopback_app testpmd testpmd
4.2.5.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.51
%system 1.08
%guest 0.00
%CPU 1.56
CPU
minflt/s 0.15
majflt/s 0.00
VSZ 3364224
RSS 8133
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44316
RSS 5804
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.06
%system 0.17
%guest 199.95
%CPU 200.17
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2843168
RSS 67496
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.51
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.06
%system 0.16
%guest 199.96
%CPU 200.17
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2754788
RSS 63600
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.5.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.5.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 548
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:44:26
start_time 2018-05-29 09:54:10
stop_time 2018-05-29 10:38:37
guest_loopback_app testpmd testpmd
4.2.5.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.51
%system 1.08
%guest 0.00
%CPU 1.56
CPU
minflt/s 0.15
majflt/s 0.00
VSZ 3364224
RSS 8133
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44316
RSS 5804
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.06
%system 0.17
%guest 199.95
%CPU 200.17
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2843168
RSS 67496
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.51
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.06
%system 0.16
%guest 199.96
%CPU 200.17
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2754788
RSS 63600
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.5.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.5.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 744
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:44:26
start_time 2018-05-29 09:54:10
stop_time 2018-05-29 10:38:37
guest_loopback_app testpmd testpmd
4.2.5.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.51
%system 1.08
%guest 0.00
%CPU 1.56
CPU
minflt/s 0.15
majflt/s 0.00
VSZ 3364224
RSS 8133
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44316
RSS 5804
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.06
%system 0.17
%guest 199.95
%CPU 200.17
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2843168
RSS 67496
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.51
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.06
%system 0.16
%guest 199.96
%CPU 200.17
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2754788
RSS 63600
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.5.7. Anomalies

No anomalies were detected during the course of this test.

4.2.5.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.6. Test ID: PVP_BACK2BACK

4.2.6.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: c79adfe7660ffa43f597af794412c0616a785943

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • VNF: QemuVirtioNet, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-16.04_trex_20180209.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvp_back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: pvp
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
4.2.6.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.6.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 425
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:12
start_time 2018-05-29 08:41:03
stop_time 2018-05-29 09:24:16
guest_loopback_app testpmd
4.2.6.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.50
%system 1.06
%guest 0.00
%CPU 1.56
CPU
minflt/s 0.12
majflt/s 0.00
VSZ 3364108
RSS 7728
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5960
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.08
%system 0.16
%guest 199.99
%CPU 200.23
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2816317
RSS 65416
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 7.80
kB_ccwr/s 0.00
4.2.6.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.6.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 483
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:12
start_time 2018-05-29 08:41:03
stop_time 2018-05-29 09:24:16
guest_loopback_app testpmd
4.2.6.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.50
%system 1.06
%guest 0.00
%CPU 1.56
CPU
minflt/s 0.12
majflt/s 0.00
VSZ 3364108
RSS 7728
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5960
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.08
%system 0.16
%guest 199.99
%CPU 200.23
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2816317
RSS 65416
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 7.80
kB_ccwr/s 0.00
4.2.6.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.6.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 537
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:12
start_time 2018-05-29 08:41:03
stop_time 2018-05-29 09:24:16
guest_loopback_app testpmd
4.2.6.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.50
%system 1.06
%guest 0.00
%CPU 1.56
CPU
minflt/s 0.12
majflt/s 0.00
VSZ 3364108
RSS 7728
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5960
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.08
%system 0.16
%guest 199.99
%CPU 200.23
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2816317
RSS 65416
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 7.80
kB_ccwr/s 0.00
4.2.6.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.6.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 548
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:12
start_time 2018-05-29 08:41:03
stop_time 2018-05-29 09:24:16
guest_loopback_app testpmd
4.2.6.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.50
%system 1.06
%guest 0.00
%CPU 1.56
CPU
minflt/s 0.12
majflt/s 0.00
VSZ 3364108
RSS 7728
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5960
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.08
%system 0.16
%guest 199.99
%CPU 200.23
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2816317
RSS 65416
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 7.80
kB_ccwr/s 0.00
4.2.6.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.6.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 744
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:12
start_time 2018-05-29 08:41:03
stop_time 2018-05-29 09:24:16
guest_loopback_app testpmd
4.2.6.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.50
%system 1.06
%guest 0.00
%CPU 1.56
CPU
minflt/s 0.12
majflt/s 0.00
VSZ 3364108
RSS 7728
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5960
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.08
%system 0.16
%guest 199.99
%CPU 200.23
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2816317
RSS 65416
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 7.80
kB_ccwr/s 0.00
4.2.6.7. Anomalies

No anomalies were detected during the course of this test.

4.2.6.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.7. Test ID: PHY2PHY_TPUT

4.2.7.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: c79adfe7660ffa43f597af794412c0616a785943

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

Below are test details:

  • Test ID: phy2phy_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: p2p
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
4.2.7.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.7.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 758000.601
tx_rate_mbps Unknown
throughput_rx_mbps 388.096
tx_rate_percent 2.547
throughput_rx_percent 2.547
frame_loss_percent 0.000
min_latency_ns 4020.000
max_latency_ns 1149340.000
avg_latency_ns 30909.500
type rfc2544_throughput
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:43
start_time 2018-05-29 06:06:55
stop_time 2018-05-29 06:34:39
4.2.7.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.49
%system 1.09
%guest 0.00
%CPU 1.57
CPU
minflt/s 0.18
majflt/s 0.00
VSZ 3364112
RSS 7778
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5732
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.7.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.7.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 952810.421
tx_rate_mbps Unknown
throughput_rx_mbps 975.678
tx_rate_percent 5.641
throughput_rx_percent 5.641
frame_loss_percent 0.000
min_latency_ns 4200.000
max_latency_ns 2494620.000
avg_latency_ns 39315.500
type rfc2544_throughput
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:43
start_time 2018-05-29 06:06:55
stop_time 2018-05-29 06:34:39
4.2.7.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.49
%system 1.09
%guest 0.00
%CPU 1.57
CPU
minflt/s 0.18
majflt/s 0.00
VSZ 3364112
RSS 7778
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5732
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.7.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.7.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 810255.201
tx_rate_mbps Unknown
throughput_rx_mbps 3318.805
tx_rate_percent 17.242
throughput_rx_percent 17.242
frame_loss_percent 0.000
min_latency_ns 5040.000
max_latency_ns 542280.000
avg_latency_ns 30480.000
type rfc2544_throughput
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:43
start_time 2018-05-29 06:06:55
stop_time 2018-05-29 06:34:39
4.2.7.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.49
%system 1.09
%guest 0.00
%CPU 1.57
CPU
minflt/s 0.18
majflt/s 0.00
VSZ 3364112
RSS 7778
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5732
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.7.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.7.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 820350.994
tx_rate_mbps Unknown
throughput_rx_mbps 6720.315
tx_rate_percent 34.258
throughput_rx_percent 34.258
frame_loss_percent 0.000
min_latency_ns 5800.000
max_latency_ns 1262860.000
avg_latency_ns 31522.500
type rfc2544_throughput
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:43
start_time 2018-05-29 06:06:55
stop_time 2018-05-29 06:34:39
4.2.7.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.49
%system 1.09
%guest 0.00
%CPU 1.57
CPU
minflt/s 0.18
majflt/s 0.00
VSZ 3364112
RSS 7778
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5732
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.7.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.7.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 833446.067
tx_rate_mbps Unknown
throughput_rx_mbps 10121.369
tx_rate_percent 51.273
throughput_rx_percent 51.273
frame_loss_percent 0.000
min_latency_ns 7120.000
max_latency_ns 493720.000
avg_latency_ns 34658.500
type rfc2544_throughput
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:43
start_time 2018-05-29 06:06:55
stop_time 2018-05-29 06:34:39
4.2.7.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.49
%system 1.09
%guest 0.00
%CPU 1.57
CPU
minflt/s 0.18
majflt/s 0.00
VSZ 3364112
RSS 7778
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5732
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.7.7. Anomalies

No anomalies were detected during the course of this test.

4.2.7.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.8. Test ID: PVP_TPUT

4.2.8.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: c79adfe7660ffa43f597af794412c0616a785943

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • VNF: QemuVirtioNet, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-16.04_trex_20180209.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvp_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvp
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
4.2.8.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.8.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 226172.930
tx_rate_mbps Unknown
throughput_rx_mbps 115.801
tx_rate_percent 1.773
throughput_rx_percent 0.760
frame_loss_percent 57.147
min_latency_ns 20720.000
max_latency_ns 95951120.000
avg_latency_ns 5364099.000
type rfc2544_throughput
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:28:43
start_time 2018-05-29 08:12:19
stop_time 2018-05-29 08:41:03
guest_loopback_app testpmd
4.2.8.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.50
%system 0.99
%guest 0.00
%CPU 1.50
CPU
minflt/s 0.19
majflt/s 0.00
VSZ 3364108
RSS 7791
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5916
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.08
%system 1.49
%guest 198.66
%CPU 200.22
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 3032027
RSS 67929
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.05
kB_ccwr/s 0.00
4.2.8.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.8.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 299542.545
tx_rate_mbps Unknown
throughput_rx_mbps 306.732
tx_rate_percent 1.773
throughput_rx_percent 1.773
frame_loss_percent 0.008
min_latency_ns 11080.000
max_latency_ns 7266140.000
avg_latency_ns 58872.000
type rfc2544_throughput
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:28:43
start_time 2018-05-29 08:12:19
stop_time 2018-05-29 08:41:03
guest_loopback_app testpmd
4.2.8.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.50
%system 0.99
%guest 0.00
%CPU 1.50
CPU
minflt/s 0.19
majflt/s 0.00
VSZ 3364108
RSS 7791
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5916
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.08
%system 1.49
%guest 198.66
%CPU 200.22
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 3032027
RSS 67929
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.05
kB_ccwr/s 0.00
4.2.8.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.8.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 265067.215
tx_rate_mbps Unknown
throughput_rx_mbps 1085.715
tx_rate_percent 5.641
throughput_rx_percent 5.641
frame_loss_percent 0.000
min_latency_ns 14940.000
max_latency_ns 654960.000
avg_latency_ns 73568.500
type rfc2544_throughput
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:28:43
start_time 2018-05-29 08:12:19
stop_time 2018-05-29 08:41:03
guest_loopback_app testpmd
4.2.8.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.50
%system 0.99
%guest 0.00
%CPU 1.50
CPU
minflt/s 0.19
majflt/s 0.00
VSZ 3364108
RSS 7791
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5916
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.08
%system 1.49
%guest 198.66
%CPU 200.22
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 3032027
RSS 67929
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.05
kB_ccwr/s 0.00
4.2.8.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.8.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 264719.935
tx_rate_mbps Unknown
throughput_rx_mbps 2168.586
tx_rate_percent 11.055
throughput_rx_percent 11.055
frame_loss_percent 0.000
min_latency_ns 16740.000
max_latency_ns 908480.000
avg_latency_ns 76093.500
type rfc2544_throughput
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:28:43
start_time 2018-05-29 08:12:19
stop_time 2018-05-29 08:41:03
guest_loopback_app testpmd
4.2.8.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.50
%system 0.99
%guest 0.00
%CPU 1.50
CPU
minflt/s 0.19
majflt/s 0.00
VSZ 3364108
RSS 7791
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5916
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.08
%system 1.49
%guest 198.66
%CPU 200.22
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 3032027
RSS 67929
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.05
kB_ccwr/s 0.00
4.2.8.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.8.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 267697.541
tx_rate_mbps Unknown
throughput_rx_mbps 3250.919
tx_rate_percent 16.469
throughput_rx_percent 16.469
frame_loss_percent 0.000
min_latency_ns 18980.000
max_latency_ns 939380.000
avg_latency_ns 77904.000
type rfc2544_throughput
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:28:43
start_time 2018-05-29 08:12:19
stop_time 2018-05-29 08:41:03
guest_loopback_app testpmd
4.2.8.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.50
%system 0.99
%guest 0.00
%CPU 1.50
CPU
minflt/s 0.19
majflt/s 0.00
VSZ 3364108
RSS 7791
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5916
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.08
%system 1.49
%guest 198.66
%CPU 200.22
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 3032027
RSS 67929
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.05
kB_ccwr/s 0.00
4.2.8.7. Anomalies

No anomalies were detected during the course of this test.

4.2.8.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.3. Rationale for decisions

The tests conducted do not have pass/fail/conditional-pass criteria. The test is simply conducted and the results are reported.

4.4. Conclusions and recommendations

The test results are stable. The vsperf CI jobs that were used to obtain the results can be found at https://artifacts.opnfv.org/logs/vswitchperf/intel-pod12/2018-05-29_10-09-09/vswitchperf_logs_2018-05-29_10-09-09.tar.gz.

4.5. General

4.5.1. Glossary

  • NFV - Network Function Virtualization
  • Mbps - 1,000,000bps

4.5.2. Document change procedures and history

Document ID Author Date Modified
OPNFV_vswitchperf_LTR_ver_1.0_Jan_15_CN_DRAFT Christopher Nolan 23/01/2015
OPNFV_vswitchperf_LTR_ver_1.1_Jan_15_CN_DRAFT Christopher Nolan 28/01/2015

5. Performance report for VppDpdkVhost

5.1. Introduction

The objective of the OPNFV project titled “Characterise vSwitch Performance for Telco NFV Use Cases”, is to evaluate a virtual switch to identify its suitability for a Telco Network Function Virtualization (NFV) environment. As well as this, the project aims to identify any gaps or bottlenecks in order to drive architectural changes to improve virtual switch performance and determinism. The purpose of this document is to summarize the results of the tests carried out on the virtual switch in the Network Function Virtualization Infrastructure (NFVI) and, from these results, provide evaluations and recommendations for the virtual switch. Test results will be outlined in details-of-LTR, preceded by the document-identifier and the scope and references).

This document is currently in draft form.

5.1.1. Document identifier

The document id will be used to uniquely identify versions of the LTR. The format for the document id will be: OPNFV_vswitchperf_LTR_rel_STATUS, the status is one of: DRAFT, REVIEWED, CORRECTED or FINAL. The document id for this version of the LTR is: OPNFV_vswitchperf_LTR_Brahmaputra_DRAFT.

5.1.2. Scope

The scope of this report is to detail the results of the tests that have been performed on the virtual switch. This report will also evaluate the results of these tests and, based on these evaluations, provide recommendations on the suitability of the virtual switch for use in a Telco NFV environment.

5.2. Details of the Level Test Report

This section provides a test-results-overview. Also included are the rationale and the conclusions.

5.2.1. Test ID: PVP_TPUT_VPP

5.2.1.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: clean

  • vswitchperf: GIT tag: c79adfe7660ffa43f597af794412c0616a785943

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: VppDpdkVhost, Version: v17.07-release, GIT tag: NA

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvp_tput_vpp
  • Description: VPP: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: clean
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
5.2.1.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

5.2.1.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 3980657.317
tx_rate_mbps Unknown
throughput_rx_mbps 2038.097
tx_rate_percent 13.375
throughput_rx_percent 13.375
frame_loss_percent 0.000
min_latency_ns 7540.000
max_latency_ns 48760.000
avg_latency_ns 13330.500
type rfc2544_throughput
packet_size 64
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:25:10
start_time 2018-05-29 11:30:20
stop_time 2018-05-29 11:55:30
5.2.1.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 601.03
%system 0.29
%guest 0.00
%CPU 601.32
CPU
minflt/s 0.12
majflt/s 0.00
VSZ 9767047
RSS 51122
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.1.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

5.2.1.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4349665.393
tx_rate_mbps Unknown
throughput_rx_mbps 4454.057
tx_rate_percent 25.750
throughput_rx_percent 25.750
frame_loss_percent 0.000
min_latency_ns 8380.000
max_latency_ns 48380.000
avg_latency_ns 16496.000
type rfc2544_throughput
packet_size 128
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:25:10
start_time 2018-05-29 11:30:20
stop_time 2018-05-29 11:55:30
5.2.1.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 601.03
%system 0.29
%guest 0.00
%CPU 601.32
CPU
minflt/s 0.12
majflt/s 0.00
VSZ 9767047
RSS 51122
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.1.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

5.2.1.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4117702.294
tx_rate_mbps Unknown
throughput_rx_mbps 16866.109
tx_rate_percent 87.625
throughput_rx_percent 87.625
frame_loss_percent 0.000
min_latency_ns 10480.000
max_latency_ns 75800.000
avg_latency_ns 36973.500
type rfc2544_throughput
packet_size 512
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:25:10
start_time 2018-05-29 11:30:20
stop_time 2018-05-29 11:55:30
5.2.1.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 601.03
%system 0.29
%guest 0.00
%CPU 601.32
CPU
minflt/s 0.12
majflt/s 0.00
VSZ 9767047
RSS 51122
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.1.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

5.2.1.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394564.047
tx_rate_mbps Unknown
throughput_rx_mbps 19616.269
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 8760.000
max_latency_ns 103980.000
avg_latency_ns 89375.000
type rfc2544_throughput
packet_size 1024
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:25:10
start_time 2018-05-29 11:30:20
stop_time 2018-05-29 11:55:30
5.2.1.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 601.03
%system 0.29
%guest 0.00
%CPU 601.32
CPU
minflt/s 0.12
majflt/s 0.00
VSZ 9767047
RSS 51122
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.1.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

5.2.1.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625445.294
tx_rate_mbps Unknown
throughput_rx_mbps 19739.408
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 12140.000
max_latency_ns 95460.000
avg_latency_ns 80461.000
type rfc2544_throughput
packet_size 1518
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:25:10
start_time 2018-05-29 11:30:20
stop_time 2018-05-29 11:55:30
5.2.1.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 601.03
%system 0.29
%guest 0.00
%CPU 601.32
CPU
minflt/s 0.12
majflt/s 0.00
VSZ 9767047
RSS 51122
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.1.7. Anomalies

No anomalies were detected during the course of this test.

5.2.1.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

5.2.2. Test ID: PVP_BACK2BACK_VPP

5.2.2.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: clean

  • vswitchperf: GIT tag: c79adfe7660ffa43f597af794412c0616a785943

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: VppDpdkVhost, Version: v17.07-release, GIT tag: NA

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvp_back2back_vpp
  • Description: VPP: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: clean
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
5.2.2.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

5.2.2.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544_back2back
packet_size 64
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:45:48
start_time 2018-05-29 11:55:31
stop_time 2018-05-29 12:41:19
5.2.2.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.60
%system 0.30
%guest 0.00
%CPU 600.90
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 9971491
RSS 50913
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.2.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

5.2.2.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 483
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 128
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:45:48
start_time 2018-05-29 11:55:31
stop_time 2018-05-29 12:41:19
5.2.2.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.60
%system 0.30
%guest 0.00
%CPU 600.90
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 9971491
RSS 50913
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.2.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

5.2.2.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 70488721
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:45:48
start_time 2018-05-29 11:55:31
stop_time 2018-05-29 12:41:19
5.2.2.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.60
%system 0.30
%guest 0.00
%CPU 600.90
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 9971491
RSS 50913
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.2.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

5.2.2.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 35919540
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:45:48
start_time 2018-05-29 11:55:31
stop_time 2018-05-29 12:41:19
5.2.2.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.60
%system 0.30
%guest 0.00
%CPU 600.90
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 9971491
RSS 50913
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.2.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

5.2.2.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 24382314
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:45:48
start_time 2018-05-29 11:55:31
stop_time 2018-05-29 12:41:19
5.2.2.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.60
%system 0.30
%guest 0.00
%CPU 600.90
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 9971491
RSS 50913
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.2.7. Anomalies

No anomalies were detected during the course of this test.

5.2.2.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

5.2.3. Test ID: PVVP_BACK2BACK_VPP

5.2.3.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: clean

  • vswitchperf: GIT tag: c79adfe7660ffa43f597af794412c0616a785943

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: VppDpdkVhost, Version: v17.07-release, GIT tag: NA

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvvp_back2back_vpp
  • Description: VPP: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: clean
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
5.2.3.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

5.2.3.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544_back2back
packet_size 64
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:51:43
start_time 2018-05-29 12:46:57
stop_time 2018-05-29 13:38:41
5.2.3.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.72
%system 0.30
%guest 0.00
%CPU 601.02
CPU
minflt/s 0.07
majflt/s 0.00
VSZ 15767390
RSS 50956
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.3.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

5.2.3.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544_back2back
packet_size 128
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:51:43
start_time 2018-05-29 12:46:57
stop_time 2018-05-29 13:38:41
5.2.3.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.72
%system 0.30
%guest 0.00
%CPU 601.02
CPU
minflt/s 0.07
majflt/s 0.00
VSZ 15767390
RSS 50956
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.3.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

5.2.3.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1075
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:51:43
start_time 2018-05-29 12:46:57
stop_time 2018-05-29 13:38:41
5.2.3.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.72
%system 0.30
%guest 0.00
%CPU 601.02
CPU
minflt/s 0.07
majflt/s 0.00
VSZ 15767390
RSS 50956
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.3.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

5.2.3.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 35919532
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:51:43
start_time 2018-05-29 12:46:57
stop_time 2018-05-29 13:38:41
5.2.3.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.72
%system 0.30
%guest 0.00
%CPU 601.02
CPU
minflt/s 0.07
majflt/s 0.00
VSZ 15767390
RSS 50956
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.3.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

5.2.3.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 24382314
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:51:43
start_time 2018-05-29 12:46:57
stop_time 2018-05-29 13:38:41
5.2.3.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.72
%system 0.30
%guest 0.00
%CPU 601.02
CPU
minflt/s 0.07
majflt/s 0.00
VSZ 15767390
RSS 50956
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.3.7. Anomalies

No anomalies were detected during the course of this test.

5.2.3.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

5.2.4. Test ID: PHY2PHY_BACK2BACK_VPP

5.2.4.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: clean

  • vswitchperf: GIT tag: c79adfe7660ffa43f597af794412c0616a785943

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: VppDpdkVhost, Version: v17.07-release, GIT tag: NA

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: phy2phy_back2back_vpp
  • Description: VPP: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: clean
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
5.2.4.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

5.2.4.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 23841
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 64
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:30:23
start_time 2018-05-29 10:59:56
stop_time 2018-05-29 11:30:20
5.2.4.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.51
%system 0.30
%guest 0.00
%CPU 600.81
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 3937580
RSS 50238
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.4.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

5.2.4.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 247439
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 128
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:30:23
start_time 2018-05-29 10:59:56
stop_time 2018-05-29 11:30:20
5.2.4.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.51
%system 0.30
%guest 0.00
%CPU 600.81
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 3937580
RSS 50238
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.4.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

5.2.4.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 70488721
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:30:23
start_time 2018-05-29 10:59:56
stop_time 2018-05-29 11:30:20
5.2.4.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.51
%system 0.30
%guest 0.00
%CPU 600.81
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 3937580
RSS 50238
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.4.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

5.2.4.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 35919540
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:30:23
start_time 2018-05-29 10:59:56
stop_time 2018-05-29 11:30:20
5.2.4.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.51
%system 0.30
%guest 0.00
%CPU 600.81
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 3937580
RSS 50238
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.4.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

5.2.4.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 24382314
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:30:23
start_time 2018-05-29 10:59:56
stop_time 2018-05-29 11:30:20
5.2.4.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.51
%system 0.30
%guest 0.00
%CPU 600.81
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 3937580
RSS 50238
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.4.7. Anomalies

No anomalies were detected during the course of this test.

5.2.4.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

5.2.5. Test ID: PHY2PHY_TPUT_VPP

5.2.5.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: clean

  • vswitchperf: GIT tag: c79adfe7660ffa43f597af794412c0616a785943

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: VppDpdkVhost, Version: v17.07-release, GIT tag: NA

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: phy2phy_tput_vpp
  • Description: VPP: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: clean
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
5.2.5.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

5.2.5.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 22856314.360
tx_rate_mbps Unknown
throughput_rx_mbps 11702.433
tx_rate_percent 76.797
throughput_rx_percent 76.797
frame_loss_percent 0.000
min_latency_ns 5100.000
max_latency_ns 51620.000
avg_latency_ns 22460.500
type rfc2544_throughput
packet_size 64
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:21:08
start_time 2018-05-29 10:38:47
stop_time 2018-05-29 10:59:56
5.2.5.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.75
%system 0.30
%guest 0.00
%CPU 601.05
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 3937580
RSS 50931
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.5.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

5.2.5.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 16630607.984
tx_rate_mbps Unknown
throughput_rx_mbps 17029.743
tx_rate_percent 98.453
throughput_rx_percent 98.453
frame_loss_percent 0.000
min_latency_ns 5440.000
max_latency_ns 31700.000
avg_latency_ns 12823.000
type rfc2544_throughput
packet_size 128
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:21:08
start_time 2018-05-29 10:38:47
stop_time 2018-05-29 10:59:56
5.2.5.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.75
%system 0.30
%guest 0.00
%CPU 601.05
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 3937580
RSS 50931
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.5.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

5.2.5.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699210.131
tx_rate_mbps Unknown
throughput_rx_mbps 19247.965
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 5500.000
max_latency_ns 29960.000
avg_latency_ns 19263.000
type rfc2544_throughput
packet_size 512
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:21:08
start_time 2018-05-29 10:38:47
stop_time 2018-05-29 10:59:56
5.2.5.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.75
%system 0.30
%guest 0.00
%CPU 601.05
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 3937580
RSS 50931
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.5.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

5.2.5.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394618.092
tx_rate_mbps Unknown
throughput_rx_mbps 19616.711
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 5960.000
max_latency_ns 28620.000
avg_latency_ns 18340.500
type rfc2544_throughput
packet_size 1024
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:21:08
start_time 2018-05-29 10:38:47
stop_time 2018-05-29 10:59:56
5.2.5.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.75
%system 0.30
%guest 0.00
%CPU 601.05
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 3937580
RSS 50931
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.5.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

5.2.5.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625473.484
tx_rate_mbps Unknown
throughput_rx_mbps 19739.750
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 6780.000
max_latency_ns 33480.000
avg_latency_ns 22422.500
type rfc2544_throughput
packet_size 1518
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:21:08
start_time 2018-05-29 10:38:47
stop_time 2018-05-29 10:59:56
5.2.5.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.75
%system 0.30
%guest 0.00
%CPU 601.05
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 3937580
RSS 50931
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.5.7. Anomalies

No anomalies were detected during the course of this test.

5.2.5.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

5.3. Rationale for decisions

The tests conducted do not have pass/fail/conditional-pass criteria. The test is simply conducted and the results are reported.

5.4. Conclusions and recommendations

The test results are stable. The vsperf CI jobs that were used to obtain the results can be found at https://artifacts.opnfv.org/logs/vswitchperf/intel-pod12/2018-05-29_10-09-09/vswitchperf_logs_2018-05-29_10-09-09.tar.gz.

5.5. General

5.5.1. Glossary

  • NFV - Network Function Virtualization
  • Mbps - 1,000,000bps

5.5.2. Document change procedures and history

Document ID Author Date Modified
OPNFV_vswitchperf_LTR_ver_1.0_Jan_15_CN_DRAFT Christopher Nolan 23/01/2015
OPNFV_vswitchperf_LTR_ver_1.1_Jan_15_CN_DRAFT Christopher Nolan 28/01/2015

6. Performance report for none

6.1. Introduction

The objective of the OPNFV project titled “Characterise vSwitch Performance for Telco NFV Use Cases”, is to evaluate a virtual switch to identify its suitability for a Telco Network Function Virtualization (NFV) environment. As well as this, the project aims to identify any gaps or bottlenecks in order to drive architectural changes to improve virtual switch performance and determinism. The purpose of this document is to summarize the results of the tests carried out on the virtual switch in the Network Function Virtualization Infrastructure (NFVI) and, from these results, provide evaluations and recommendations for the virtual switch. Test results will be outlined in details-of-LTR, preceded by the document-identifier and the scope and references).

This document is currently in draft form.

6.1.1. Document identifier

The document id will be used to uniquely identify versions of the LTR. The format for the document id will be: OPNFV_vswitchperf_LTR_rel_STATUS, the status is one of: DRAFT, REVIEWED, CORRECTED or FINAL. The document id for this version of the LTR is: OPNFV_vswitchperf_LTR_Brahmaputra_DRAFT.

6.1.2. Scope

The scope of this report is to detail the results of the tests that have been performed on the virtual switch. This report will also evaluate the results of these tests and, based on these evaluations, provide recommendations on the suitability of the virtual switch for use in a Telco NFV environment.

6.2. Details of the Level Test Report

This section provides a test-results-overview. Also included are the rationale and the conclusions.

6.2.1. Test ID: PVP_TPUT

6.2.1.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
    • Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: c79adfe7660ffa43f597af794412c0616a785943

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: none, Version: None, GIT tag: None

  • VNF: QemuPciPassthrough, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvp_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvp
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
6.2.1.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

6.2.1.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 7893901.110
tx_rate_mbps Unknown
throughput_rx_mbps 4041.677
tx_rate_percent 26.523
throughput_rx_percent 26.523
frame_loss_percent 0.000
min_latency_ns 3580.000
max_latency_ns 26460.000
avg_latency_ns 3931.000
type rfc2544_throughput
packet_size 64
vswitch none
traffic_type udp
test_execution_time 00:21:55
start_time 2018-05-29 13:38:54
stop_time 2018-05-29 14:00:49
guest_loopback_app testpmd
6.2.1.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.01
%system 0.02
%guest 200.02
%CPU 200.05
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2839496
RSS 76235
%MEM 0.12
kB_rd/s 0.00
kB_wr/s 17.43
kB_ccwr/s 0.00
6.2.1.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

6.2.1.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 7877165.526
tx_rate_mbps Unknown
throughput_rx_mbps 8066.217
tx_rate_percent 46.633
throughput_rx_percent 46.633
frame_loss_percent 0.000
min_latency_ns 3780.000
max_latency_ns 26740.000
avg_latency_ns 4195.500
type rfc2544_throughput
packet_size 128
vswitch none
traffic_type udp
test_execution_time 00:21:55
start_time 2018-05-29 13:38:54
stop_time 2018-05-29 14:00:49
guest_loopback_app testpmd
6.2.1.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.01
%system 0.02
%guest 200.02
%CPU 200.05
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2839496
RSS 76235
%MEM 0.12
kB_rd/s 0.00
kB_wr/s 17.43
kB_ccwr/s 0.00
6.2.1.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

6.2.1.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699191.100
tx_rate_mbps Unknown
throughput_rx_mbps 19247.887
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 4880.000
max_latency_ns 44040.000
avg_latency_ns 27777.000
type rfc2544_throughput
packet_size 512
vswitch none
traffic_type udp
test_execution_time 00:21:55
start_time 2018-05-29 13:38:54
stop_time 2018-05-29 14:00:49
guest_loopback_app testpmd
6.2.1.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.01
%system 0.02
%guest 200.02
%CPU 200.05
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2839496
RSS 76235
%MEM 0.12
kB_rd/s 0.00
kB_wr/s 17.43
kB_ccwr/s 0.00
6.2.1.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

6.2.1.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394606.861
tx_rate_mbps Unknown
throughput_rx_mbps 19616.619
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 5860.000
max_latency_ns 42880.000
avg_latency_ns 29709.000
type rfc2544_throughput
packet_size 1024
vswitch none
traffic_type udp
test_execution_time 00:21:55
start_time 2018-05-29 13:38:54
stop_time 2018-05-29 14:00:49
guest_loopback_app testpmd
6.2.1.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.01
%system 0.02
%guest 200.02
%CPU 200.05
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2839496
RSS 76235
%MEM 0.12
kB_rd/s 0.00
kB_wr/s 17.43
kB_ccwr/s 0.00
6.2.1.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

6.2.1.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625470.233
tx_rate_mbps Unknown
throughput_rx_mbps 19739.711
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 5980.000
max_latency_ns 40860.000
avg_latency_ns 26189.000
type rfc2544_throughput
packet_size 1518
vswitch none
traffic_type udp
test_execution_time 00:21:55
start_time 2018-05-29 13:38:54
stop_time 2018-05-29 14:00:49
guest_loopback_app testpmd
6.2.1.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.01
%system 0.02
%guest 200.02
%CPU 200.05
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2839496
RSS 76235
%MEM 0.12
kB_rd/s 0.00
kB_wr/s 17.43
kB_ccwr/s 0.00
6.2.1.7. Anomalies

No anomalies were detected during the course of this test.

6.2.1.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

6.3. Rationale for decisions

The tests conducted do not have pass/fail/conditional-pass criteria. The test is simply conducted and the results are reported.

6.4. Conclusions and recommendations

The test results are stable. The vsperf CI jobs that were used to obtain the results can be found at https://artifacts.opnfv.org/logs/vswitchperf/intel-pod12/2018-05-29_10-09-09/vswitchperf_logs_2018-05-29_10-09-09.tar.gz.

6.5. General

6.5.1. Glossary

  • NFV - Network Function Virtualization
  • Mbps - 1,000,000bps

6.5.2. Document change procedures and history

Document ID Author Date Modified
OPNFV_vswitchperf_LTR_ver_1.0_Jan_15_CN_DRAFT Christopher Nolan 23/01/2015
OPNFV_vswitchperf_LTR_ver_1.1_Jan_15_CN_DRAFT Christopher Nolan 28/01/2015