VSPERF Results

1. OPNFV Brahmaputra Scenarios

Available Tests and aspects of scenarios:

Framework Test Definition
phy2phy_tput PacketLossRatio for Phy2Phy
back2back BackToBackFrames for Phy2Phy
phy2phy_tput_mod_vlan PacketLossRatioFrameModification for Phy2Phy
phy2phy_cont Phy2Phy blast vswitch at x% TX rate and measure throughput
pvp_cont PVP blast vswitch at x% TX rate and measure throughput
pvvp_cont PVVP blast vswitch at x% TX rate and measure throughput
phy2phy_scalability Scalability0PacketLoss for Phy2Phy
pvp_tput PacketLossRatio for PVP
pvp_back2back BackToBackFrames for PVP
pvvp_tput PacketLossRatio for PVVP
pvvp_back2back BackToBackFrames for PVVP
phy2phy_cpu_load CPU0PacketLoss for Phy2Phy
phy2phy_mem_load Same as CPU0PacketLoss but using a memory intensive app

Supported deployment scenarios:

  • Phy2Phy: Physical port -> vSwitch -> Physical port.
  • PVP: Physical port -> vSwitch -> VNF -> vSwitch -> Physical port.
  • PVVP: Physical port -> vSwitch -> VNF -> vSwitch -> VNF -> vSwitch -> Physical port.

Loopback applications in the Guest can be:

Supported traffic generators:

  • Ixia: IxOS and IxNet.
  • Spirent.
  • Dummy.

2. OPNFV Brahmaputra Results

The vsperf CI jobs that were used to obtain the results can be found at https://wiki.opnfv.org/wiki/vsperf_results.

The following table maps the results in the test dashboard to the appropriate test case in the VSPERF Framework and specifies the metric the vertical/Y axis is plotting. Please note, the presence of dpdk within a test name signifies that the vswitch under test was OVS with DPDK, while its absence indicates that the vswitch under test was stock OVS.

Dashboard Test Framework Test Metric Guest Interface
tput_ovsdpdk phy2phy_tput Throughput (FPS) N/A
tput_ovs phy2phy_tput Throughput (FPS) N/A
b2b_ovsdpdk back2back Back-to-back value N/A
b2b_ovs back2back Back-to-back value N/A
tput_mod_vlan_ovs phy2phy_tput_mod_vlan Throughput (FPS) N/A
tput_mod_vlan_ovsdpdk phy2phy_tput_mod_vlan Throughput (FPS) N/A
scalability_ovs phy2phy_scalability Throughput (FPS) N/A
scalability_ovsdpdk phy2phy_scalability Throughput (FPS) N/A
pvp_tput_ovsdpdkuser pvp_tput Throughput (FPS) vhost-user
pvp_tput_ovsvirtio pvp_tput Throughput (FPS) virtio-net
pvp_b2b_ovsdpdkuser pvp_back2back Back-to-back value vhost-user
pvp_b2b_ovsvirtio pvp_back2back Back-to-back value virtio-net
pvvp_tput_ovsdpdkuser pvvp_tput Throughput (FPS) vhost-user
pvvp_tput_ovsvirtio pvvp_tput Throughput (FPS) virtio-net
pvvp_b2b_ovsdpdkuser pvvp_back2back Throughput (FPS) vhost-user
pvvp_b2b_ovsvirtio pvvp_back2back Throughput (FPS) virtio-net

The loopback application in the VNF used for PVP and PVVP scenarios was DPDK testpmd.

3. Performance report for Open vSwitch with DPDK support

3.1. Introduction

The objective of the OPNFV project titled “Characterise vSwitch Performance for Telco NFV Use Cases”, is to evaluate a virtual switch to identify its suitability for a Telco Network Function Virtualization (NFV) environment. As well as this, the project aims to identify any gaps or bottlenecks in order to drive architectural changes to improve virtual switch performance and determinism. The purpose of this document is to summarize the results of the tests carried out on the virtual switch in the Network Function Virtualization Infrastructure (NFVI) and, from these results, provide evaluations and recommendations for the virtual switch. Test results will be outlined in details-of-LTR, preceded by the document-identifier and the scope and references).

This document is currently in draft form.

3.1.1. Document identifier

The document id will be used to uniquely identify versions of the LTR. The format for the document id will be: OPNFV_vswitchperf_LTR_rel_STATUS, the status is one of: DRAFT, REVIEWED, CORRECTED or FINAL. The document id for this version of the LTR is: OPNFV_vswitchperf_LTR_Brahmaputra_DRAFT.

3.1.2. Scope

The scope of this report is to detail the results of the tests that have been performed on the virtual switch. This report will also evaluate the results of these tests and, based on these evaluations, provide recommendations on the suitability of the virtual switch for use in a Telco NFV environment.

3.2. Details of the Level Test Report

This section provides a test-results-overview. Also included are the rationale and the conclusions.

3.2.1. Test ID: BACK2BACK

3.2.1.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 513153802b329c0584d72655460a4a264ad41e7c

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • DPDK Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: p2p
  • Traffic type: rfc2544
  • Bidirectional : True
3.2.1.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.1.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 23416
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:34:17
3.2.1.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 19013
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 19.76
majflt/s 0.00
VSZ 8057820
RSS 27164
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 18997
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52080
RSS 3456
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.1.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 63344594
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:34:17
3.2.1.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 19013
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 19.76
majflt/s 0.00
VSZ 8057820
RSS 27164
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 18997
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52080
RSS 3456
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.1.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 70488721
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:34:17
3.2.1.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 19013
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 19.76
majflt/s 0.00
VSZ 8057820
RSS 27164
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 18997
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52080
RSS 3456
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.1.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 35919540
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:34:17
3.2.1.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 19013
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 19.76
majflt/s 0.00
VSZ 8057820
RSS 27164
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 18997
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52080
RSS 3456
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.1.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 24382314
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:34:17
3.2.1.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 19013
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 19.76
majflt/s 0.00
VSZ 8057820
RSS 27164
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 18997
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52080
RSS 3456
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.7. Anomalies

No anomalies were detected during the course of this test.

3.2.1.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.2. Test ID: PVP_TPUT

3.2.2.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: 513153802b329c0584d72655460a4a264ad41e7c

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • DPDK Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

  • VNF: QemuDpdkVhostUser, Version: 2.5.0, GIT tag: a8c40fa2d667e585382080db36ac44e216b37a1c

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: pvp_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvp
  • Traffic type: rfc2544
  • Bidirectional : True
3.2.2.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.2.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4671230.193
tx_rate_mbps Unknown
throughput_rx_mbps 2391.670
tx_rate_percent 15.695
throughput_rx_percent 15.695
frame_loss_percent 0.000
min_latency_ns 4920.000
max_latency_ns 79480.000
avg_latency_ns 6499.000
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:26:41
guest_loopback_app testpmd
3.2.2.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 31088
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 16.38
majflt/s 0.00
VSZ 12256216
RSS 25309
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 30970
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52076
RSS 3496
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 31331
%usr 0.00
%system 0.00
%guest 200.10
%CPU 100.16
CPU
minflt/s 0.60
majflt/s 0.00
VSZ 2652560
RSS 109742
%MEM 0.17
kB_rd/s 0.00
kB_wr/s 9.24
kB_ccwr/s 0.00
3.2.2.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.2.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 3043181.566
tx_rate_mbps Unknown
throughput_rx_mbps 3116.218
tx_rate_percent 18.016
throughput_rx_percent 18.016
frame_loss_percent 0.000
min_latency_ns 4620.000
max_latency_ns 104880.000
avg_latency_ns 5980.000
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:26:41
guest_loopback_app testpmd
3.2.2.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 31088
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 16.38
majflt/s 0.00
VSZ 12256216
RSS 25309
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 30970
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52076
RSS 3496
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 31331
%usr 0.00
%system 0.00
%guest 200.10
%CPU 100.16
CPU
minflt/s 0.60
majflt/s 0.00
VSZ 2652560
RSS 109742
%MEM 0.17
kB_rd/s 0.00
kB_wr/s 9.24
kB_ccwr/s 0.00
3.2.2.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.2.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699063.882
tx_rate_mbps Unknown
throughput_rx_mbps 19247.366
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 23480.000
max_latency_ns 147420.000
avg_latency_ns 136043.500
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:26:41
guest_loopback_app testpmd
3.2.2.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 31088
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 16.38
majflt/s 0.00
VSZ 12256216
RSS 25309
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 30970
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52076
RSS 3496
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 31331
%usr 0.00
%system 0.00
%guest 200.10
%CPU 100.16
CPU
minflt/s 0.60
majflt/s 0.00
VSZ 2652560
RSS 109742
%MEM 0.17
kB_rd/s 0.00
kB_wr/s 9.24
kB_ccwr/s 0.00
3.2.2.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.2.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394581.647
tx_rate_mbps Unknown
throughput_rx_mbps 19616.413
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 60420.000
max_latency_ns 146340.000
avg_latency_ns 125052.000
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:26:41
guest_loopback_app testpmd
3.2.2.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 31088
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 16.38
majflt/s 0.00
VSZ 12256216
RSS 25309
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 30970
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52076
RSS 3496
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 31331
%usr 0.00
%system 0.00
%guest 200.10
%CPU 100.16
CPU
minflt/s 0.60
majflt/s 0.00
VSZ 2652560
RSS 109742
%MEM 0.17
kB_rd/s 0.00
kB_wr/s 9.24
kB_ccwr/s 0.00
3.2.2.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.2.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625453.226
tx_rate_mbps Unknown
throughput_rx_mbps 19739.504
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 58680.000
max_latency_ns 139400.000
avg_latency_ns 121224.000
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:26:41
guest_loopback_app testpmd
3.2.2.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 31088
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 16.38
majflt/s 0.00
VSZ 12256216
RSS 25309
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 30970
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52076
RSS 3496
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 31331
%usr 0.00
%system 0.00
%guest 200.10
%CPU 100.16
CPU
minflt/s 0.60
majflt/s 0.00
VSZ 2652560
RSS 109742
%MEM 0.17
kB_rd/s 0.00
kB_wr/s 9.24
kB_ccwr/s 0.00
3.2.2.7. Anomalies

No anomalies were detected during the course of this test.

3.2.2.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.3. Test ID: PHY2PHY_TPUT_MOD_VLAN

3.2.3.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 513153802b329c0584d72655460a4a264ad41e7c

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • DPDK Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: phy2phy_tput_mod_vlan
  • Description: LTD.Throughput.RFC2544.PacketLossRatioFrameModification
  • Deployment: p2p
  • Traffic type: rfc2544
  • Bidirectional : False
3.2.3.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.3.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 12233820.568
tx_rate_mbps Unknown
throughput_rx_mbps 6655.198
tx_rate_percent 82.211
throughput_rx_percent 86.126
frame_loss_percent 0.000
min_latency_ns 6920.000
max_latency_ns 343920.000
avg_latency_ns 9802.000
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:30:58
3.2.3.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 23597
%usr 200.32
%system 0.12
%guest 0.00
%CPU 200.44
CPU
minflt/s 24.30
majflt/s 0.00
VSZ 8057816
RSS 25373
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 23581
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52088
RSS 3436
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.3.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.3.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 8184666.895
tx_rate_mbps Unknown
throughput_rx_mbps 8643.008
tx_rate_percent 96.906
throughput_rx_percent 99.526
frame_loss_percent 0.000
min_latency_ns 5400.000
max_latency_ns 57180.000
avg_latency_ns 7598.000
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:30:58
3.2.3.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 23597
%usr 200.32
%system 0.12
%guest 0.00
%CPU 200.44
CPU
minflt/s 24.30
majflt/s 0.00
VSZ 8057816
RSS 25373
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 23581
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52088
RSS 3436
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.3.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.3.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2295108.410
tx_rate_mbps Unknown
throughput_rx_mbps 9474.208
tx_rate_percent 97.680
throughput_rx_percent 98.414
frame_loss_percent 0.000
min_latency_ns 5000.000
max_latency_ns 34060.000
avg_latency_ns 5905.000
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:30:58
3.2.3.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 23597
%usr 200.32
%system 0.12
%guest 0.00
%CPU 200.44
CPU
minflt/s 24.30
majflt/s 0.00
VSZ 8057816
RSS 25373
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 23581
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52088
RSS 3436
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.3.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.3.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1188058.534
tx_rate_mbps Unknown
throughput_rx_mbps 9770.593
tx_rate_percent 99.227
throughput_rx_percent 99.607
frame_loss_percent 0.000
min_latency_ns 5800.000
max_latency_ns 35040.000
avg_latency_ns 7696.000
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:30:58
3.2.3.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 23597
%usr 200.32
%system 0.12
%guest 0.00
%CPU 200.44
CPU
minflt/s 24.30
majflt/s 0.00
VSZ 8057816
RSS 25373
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 23581
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52088
RSS 3436
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.3.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.3.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 806458.397
tx_rate_mbps Unknown
throughput_rx_mbps 9819.437
tx_rate_percent 99.227
throughput_rx_percent 99.485
frame_loss_percent 0.000
min_latency_ns 6220.000
max_latency_ns 36840.000
avg_latency_ns 7751.000
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:30:58
3.2.3.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 23597
%usr 200.32
%system 0.12
%guest 0.00
%CPU 200.44
CPU
minflt/s 24.30
majflt/s 0.00
VSZ 8057816
RSS 25373
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 23581
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52088
RSS 3436
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.3.7. Anomalies

No anomalies were detected during the course of this test.

3.2.3.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.4. Test ID: PVVP_BACK2BACK

3.2.4.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: pvvp

  • vswitchperf: GIT tag: 513153802b329c0584d72655460a4a264ad41e7c

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • DPDK Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

  • VNF: QemuDpdkVhostUser, Version: 2.5.0, GIT tag: a8c40fa2d667e585382080db36ac44e216b37a1c

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: pvvp_back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: pvvp
  • Traffic type: rfc2544
  • Bidirectional : True
3.2.4.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.4.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 3405
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:55:55
guest_loopback_app testpmd testpmd
3.2.4.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 45804
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 12.01
majflt/s 0.00
VSZ 16454620
RSS 25526
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 45788
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52088
RSS 3432
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 46201
%usr 0.00
%system 0.25
%guest 200.45
%CPU 100.17
CPU
minflt/s 1.28
majflt/s 0.00
VSZ 2684173
RSS 195167
%MEM 0.30
kB_rd/s 0.00
kB_wr/s 4.73
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 46047
%usr 0.00
%system 0.22
%guest 200.43
%CPU 100.17
CPU
minflt/s 1.07
majflt/s 0.00
VSZ 2709896
RSS 211850
%MEM 0.32
kB_rd/s 0.00
kB_wr/s 0.05
kB_ccwr/s 0.00
3.2.4.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.4.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 3866
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:55:55
guest_loopback_app testpmd testpmd
3.2.4.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 45804
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 12.01
majflt/s 0.00
VSZ 16454620
RSS 25526
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 45788
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52088
RSS 3432
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 46201
%usr 0.00
%system 0.25
%guest 200.45
%CPU 100.17
CPU
minflt/s 1.28
majflt/s 0.00
VSZ 2684173
RSS 195167
%MEM 0.30
kB_rd/s 0.00
kB_wr/s 4.73
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 46047
%usr 0.00
%system 0.22
%guest 200.43
%CPU 100.17
CPU
minflt/s 1.07
majflt/s 0.00
VSZ 2709896
RSS 211850
%MEM 0.32
kB_rd/s 0.00
kB_wr/s 0.05
kB_ccwr/s 0.00
3.2.4.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.4.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 2151
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:55:55
guest_loopback_app testpmd testpmd
3.2.4.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 45804
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 12.01
majflt/s 0.00
VSZ 16454620
RSS 25526
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 45788
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52088
RSS 3432
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 46201
%usr 0.00
%system 0.25
%guest 200.45
%CPU 100.17
CPU
minflt/s 1.28
majflt/s 0.00
VSZ 2684173
RSS 195167
%MEM 0.30
kB_rd/s 0.00
kB_wr/s 4.73
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 46047
%usr 0.00
%system 0.22
%guest 200.43
%CPU 100.17
CPU
minflt/s 1.07
majflt/s 0.00
VSZ 2709896
RSS 211850
%MEM 0.32
kB_rd/s 0.00
kB_wr/s 0.05
kB_ccwr/s 0.00
3.2.4.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.4.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 24694683
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:55:55
guest_loopback_app testpmd testpmd
3.2.4.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 45804
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 12.01
majflt/s 0.00
VSZ 16454620
RSS 25526
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 45788
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52088
RSS 3432
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 46201
%usr 0.00
%system 0.25
%guest 200.45
%CPU 100.17
CPU
minflt/s 1.28
majflt/s 0.00
VSZ 2684173
RSS 195167
%MEM 0.30
kB_rd/s 0.00
kB_wr/s 4.73
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 46047
%usr 0.00
%system 0.22
%guest 200.43
%CPU 100.17
CPU
minflt/s 1.07
majflt/s 0.00
VSZ 2709896
RSS 211850
%MEM 0.32
kB_rd/s 0.00
kB_wr/s 0.05
kB_ccwr/s 0.00
3.2.4.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.4.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 18286735
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:55:55
guest_loopback_app testpmd testpmd
3.2.4.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 45804
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 12.01
majflt/s 0.00
VSZ 16454620
RSS 25526
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 45788
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52088
RSS 3432
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 46201
%usr 0.00
%system 0.25
%guest 200.45
%CPU 100.17
CPU
minflt/s 1.28
majflt/s 0.00
VSZ 2684173
RSS 195167
%MEM 0.30
kB_rd/s 0.00
kB_wr/s 4.73
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 46047
%usr 0.00
%system 0.22
%guest 200.43
%CPU 100.17
CPU
minflt/s 1.07
majflt/s 0.00
VSZ 2709896
RSS 211850
%MEM 0.32
kB_rd/s 0.00
kB_wr/s 0.05
kB_ccwr/s 0.00
3.2.4.7. Anomalies

No anomalies were detected during the course of this test.

3.2.4.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.5. Test ID: PHY2PHY_TPUT

3.2.5.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 513153802b329c0584d72655460a4a264ad41e7c

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • DPDK Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: phy2phy_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: p2p
  • Traffic type: rfc2544
  • Bidirectional : True
3.2.5.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.5.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 22626034.019
tx_rate_mbps Unknown
throughput_rx_mbps 11584.529
tx_rate_percent 76.023
throughput_rx_percent 76.023
frame_loss_percent 0.000
min_latency_ns 4660.000
max_latency_ns 384320.000
avg_latency_ns 7647.000
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:22:29
3.2.5.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 15878
%usr 200.33
%system 0.12
%guest 0.00
%CPU 200.45
CPU
minflt/s 21.69
majflt/s 0.00
VSZ 8057816
RSS 25156
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 15862
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52088
RSS 3456
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.5.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.5.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 15716125.940
tx_rate_mbps Unknown
throughput_rx_mbps 16093.313
tx_rate_percent 93.039
throughput_rx_percent 93.039
frame_loss_percent 0.000
min_latency_ns 5120.000
max_latency_ns 198120.000
avg_latency_ns 9004.500
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:22:29
3.2.5.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 15878
%usr 200.33
%system 0.12
%guest 0.00
%CPU 200.45
CPU
minflt/s 21.69
majflt/s 0.00
VSZ 8057816
RSS 25156
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 15862
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52088
RSS 3456
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.5.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.5.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699177.645
tx_rate_mbps Unknown
throughput_rx_mbps 19247.832
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 15040.000
max_latency_ns 60480.000
avg_latency_ns 47315.000
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:22:29
3.2.5.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 15878
%usr 200.33
%system 0.12
%guest 0.00
%CPU 200.45
CPU
minflt/s 21.69
majflt/s 0.00
VSZ 8057816
RSS 25156
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 15862
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52088
RSS 3456
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.5.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.5.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394608.649
tx_rate_mbps Unknown
throughput_rx_mbps 19616.634
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 46360.000
max_latency_ns 81460.000
avg_latency_ns 70183.000
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:22:29
3.2.5.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 15878
%usr 200.33
%system 0.12
%guest 0.00
%CPU 200.45
CPU
minflt/s 21.69
majflt/s 0.00
VSZ 8057816
RSS 25156
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 15862
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52088
RSS 3456
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.5.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.5.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625470.672
tx_rate_mbps Unknown
throughput_rx_mbps 19739.716
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 53740.000
max_latency_ns 85180.000
avg_latency_ns 74582.000
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:22:29
3.2.5.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 15878
%usr 200.33
%system 0.12
%guest 0.00
%CPU 200.45
CPU
minflt/s 21.69
majflt/s 0.00
VSZ 8057816
RSS 25156
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 15862
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52088
RSS 3456
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.5.7. Anomalies

No anomalies were detected during the course of this test.

3.2.5.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.6. Test ID: PHY2PHY_SCALABILITY

3.2.6.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 513153802b329c0584d72655460a4a264ad41e7c

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • DPDK Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: phy2phy_scalability
  • Description: LTD.Scalability.Flows.RFC2544.0PacketLoss
  • Deployment: p2p
  • Traffic type: rfc2544
  • Bidirectional : True
3.2.6.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.6.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 23316603.439
tx_rate_mbps Unknown
throughput_rx_mbps 11938.101
tx_rate_percent 78.344
throughput_rx_percent 78.344
frame_loss_percent 0.000
min_latency_ns 6120.000
max_latency_ns 143680.000
avg_latency_ns 9210.500
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:23:26
stream_count 8000
match_type L4
pre-installed_flows No
3.2.6.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 27717
%usr 200.32
%system 0.12
%guest 0.00
%CPU 200.44
CPU
minflt/s 24.04
majflt/s 0.00
VSZ 8057816
RSS 25351
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 27701
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52076
RSS 3482
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.6.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.6.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 15977430.516
tx_rate_mbps Unknown
throughput_rx_mbps 16360.889
tx_rate_percent 94.586
throughput_rx_percent 94.586
frame_loss_percent 0.000
min_latency_ns 6120.000
max_latency_ns 69320.000
avg_latency_ns 9182.500
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:23:26
stream_count 8000
match_type L4
pre-installed_flows No
3.2.6.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 27717
%usr 200.32
%system 0.12
%guest 0.00
%CPU 200.44
CPU
minflt/s 24.04
majflt/s 0.00
VSZ 8057816
RSS 25351
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 27701
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52076
RSS 3482
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.6.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.6.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699186.745
tx_rate_mbps Unknown
throughput_rx_mbps 19247.869
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 15100.000
max_latency_ns 55580.000
avg_latency_ns 44571.500
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:23:26
stream_count 8000
match_type L4
pre-installed_flows No
3.2.6.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 27717
%usr 200.32
%system 0.12
%guest 0.00
%CPU 200.44
CPU
minflt/s 24.04
majflt/s 0.00
VSZ 8057816
RSS 25351
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 27701
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52076
RSS 3482
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.6.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.6.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394605.066
tx_rate_mbps Unknown
throughput_rx_mbps 19616.605
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 50540.000
max_latency_ns 90020.000
avg_latency_ns 78856.500
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:23:26
stream_count 8000
match_type L4
pre-installed_flows No
3.2.6.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 27717
%usr 200.32
%system 0.12
%guest 0.00
%CPU 200.44
CPU
minflt/s 24.04
majflt/s 0.00
VSZ 8057816
RSS 25351
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 27701
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52076
RSS 3482
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.6.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.6.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625466.446
tx_rate_mbps Unknown
throughput_rx_mbps 19739.665
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 46760.000
max_latency_ns 87000.000
avg_latency_ns 75564.500
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:23:26
stream_count 8000
match_type L4
pre-installed_flows No
3.2.6.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 27717
%usr 200.32
%system 0.12
%guest 0.00
%CPU 200.44
CPU
minflt/s 24.04
majflt/s 0.00
VSZ 8057816
RSS 25351
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 27701
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52076
RSS 3482
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.6.7. Anomalies

No anomalies were detected during the course of this test.

3.2.6.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.7. Test ID: PVP_BACK2BACK

3.2.7.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: 513153802b329c0584d72655460a4a264ad41e7c

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • DPDK Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

  • VNF: QemuDpdkVhostUser, Version: 2.5.0, GIT tag: a8c40fa2d667e585382080db36ac44e216b37a1c

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: pvp_back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: pvp
  • Traffic type: rfc2544
  • Bidirectional : True
3.2.7.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.7.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 2980
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:49:46
guest_loopback_app testpmd
3.2.7.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 34794
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 17.26
majflt/s 0.00
VSZ 12256216
RSS 25389
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 34778
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52086
RSS 3497
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 35024
%usr 0.00
%system 0.00
%guest 200.10
%CPU 100.16
CPU
minflt/s 0.14
majflt/s 0.00
VSZ 2668940
RSS 194624
%MEM 0.30
kB_rd/s 0.00
kB_wr/s 3.98
kB_ccwr/s 0.00
3.2.7.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.7.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1933
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:49:46
guest_loopback_app testpmd
3.2.7.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 34794
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 17.26
majflt/s 0.00
VSZ 12256216
RSS 25389
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 34778
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52086
RSS 3497
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 35024
%usr 0.00
%system 0.00
%guest 200.10
%CPU 100.16
CPU
minflt/s 0.14
majflt/s 0.00
VSZ 2668940
RSS 194624
%MEM 0.30
kB_rd/s 0.00
kB_wr/s 3.98
kB_ccwr/s 0.00
3.2.7.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.7.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 35244360
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:49:46
guest_loopback_app testpmd
3.2.7.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 34794
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 17.26
majflt/s 0.00
VSZ 12256216
RSS 25389
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 34778
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52086
RSS 3497
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 35024
%usr 0.00
%system 0.00
%guest 200.10
%CPU 100.16
CPU
minflt/s 0.14
majflt/s 0.00
VSZ 2668940
RSS 194624
%MEM 0.30
kB_rd/s 0.00
kB_wr/s 3.98
kB_ccwr/s 0.00
3.2.7.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.7.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 35919539
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:49:46
guest_loopback_app testpmd
3.2.7.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 34794
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 17.26
majflt/s 0.00
VSZ 12256216
RSS 25389
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 34778
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52086
RSS 3497
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 35024
%usr 0.00
%system 0.00
%guest 200.10
%CPU 100.16
CPU
minflt/s 0.14
majflt/s 0.00
VSZ 2668940
RSS 194624
%MEM 0.30
kB_rd/s 0.00
kB_wr/s 3.98
kB_ccwr/s 0.00
3.2.7.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.7.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 24382314
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:49:46
guest_loopback_app testpmd
3.2.7.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 34794
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 17.26
majflt/s 0.00
VSZ 12256216
RSS 25389
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 34778
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52086
RSS 3497
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 35024
%usr 0.00
%system 0.00
%guest 200.10
%CPU 100.16
CPU
minflt/s 0.14
majflt/s 0.00
VSZ 2668940
RSS 194624
%MEM 0.30
kB_rd/s 0.00
kB_wr/s 3.98
kB_ccwr/s 0.00
3.2.7.7. Anomalies

No anomalies were detected during the course of this test.

3.2.7.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.8. Test ID: PVVP_TPUT

3.2.8.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: pvvp

  • vswitchperf: GIT tag: 513153802b329c0584d72655460a4a264ad41e7c

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • DPDK Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

  • VNF: QemuDpdkVhostUser, Version: 2.5.0, GIT tag: a8c40fa2d667e585382080db36ac44e216b37a1c

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: pvvp_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvvp
  • Traffic type: rfc2544
  • Bidirectional : True
3.2.8.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.8.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 3980657.118
tx_rate_mbps Unknown
throughput_rx_mbps 2038.096
tx_rate_percent 13.375
throughput_rx_percent 13.375
frame_loss_percent 0.000
min_latency_ns 10920.000
max_latency_ns 104320.000
avg_latency_ns 18160.000
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:33:45
guest_loopback_app testpmd testpmd
3.2.8.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 41184
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 6.04
majflt/s 0.00
VSZ 16454616
RSS 27557
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 41168
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52088
RSS 3456
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 41570
%usr 0.00
%system 0.00
%guest 200.10
%CPU 100.16
CPU
minflt/s 0.63
majflt/s 0.00
VSZ 2685324
RSS 73872
%MEM 0.11
kB_rd/s 0.00
kB_wr/s 8.09
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 41427
%usr 0.00
%system 0.01
%guest 200.13
%CPU 100.15
CPU
minflt/s 1.19
majflt/s 0.00
VSZ 2607496
RSS 101431
%MEM 0.15
kB_rd/s 0.00
kB_wr/s 0.18
kB_ccwr/s 0.00
3.2.8.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.8.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2520590.197
tx_rate_mbps Unknown
throughput_rx_mbps 2581.084
tx_rate_percent 14.922
throughput_rx_percent 14.922
frame_loss_percent 0.000
min_latency_ns 7540.000
max_latency_ns 110680.000
avg_latency_ns 10803.500
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:33:45
guest_loopback_app testpmd testpmd
3.2.8.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 41184
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 6.04
majflt/s 0.00
VSZ 16454616
RSS 27557
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 41168
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52088
RSS 3456
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 41570
%usr 0.00
%system 0.00
%guest 200.10
%CPU 100.16
CPU
minflt/s 0.63
majflt/s 0.00
VSZ 2685324
RSS 73872
%MEM 0.11
kB_rd/s 0.00
kB_wr/s 8.09
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 41427
%usr 0.00
%system 0.01
%guest 200.13
%CPU 100.15
CPU
minflt/s 1.19
majflt/s 0.00
VSZ 2607496
RSS 101431
%MEM 0.15
kB_rd/s 0.00
kB_wr/s 0.18
kB_ccwr/s 0.00
3.2.8.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.8.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 3536162.541
tx_rate_mbps Unknown
throughput_rx_mbps 14484.122
tx_rate_percent 75.250
throughput_rx_percent 75.250
frame_loss_percent 0.000
min_latency_ns 10680.000
max_latency_ns 140260.000
avg_latency_ns 35609.500
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:33:45
guest_loopback_app testpmd testpmd
3.2.8.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 41184
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 6.04
majflt/s 0.00
VSZ 16454616
RSS 27557
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 41168
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52088
RSS 3456
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 41570
%usr 0.00
%system 0.00
%guest 200.10
%CPU 100.16
CPU
minflt/s 0.63
majflt/s 0.00
VSZ 2685324
RSS 73872
%MEM 0.11
kB_rd/s 0.00
kB_wr/s 8.09
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 41427
%usr 0.00
%system 0.01
%guest 200.13
%CPU 100.15
CPU
minflt/s 1.19
majflt/s 0.00
VSZ 2607496
RSS 101431
%MEM 0.15
kB_rd/s 0.00
kB_wr/s 0.18
kB_ccwr/s 0.00
3.2.8.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.8.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2098294.591
tx_rate_mbps Unknown
throughput_rx_mbps 17189.229
tx_rate_percent 87.625
throughput_rx_percent 87.625
frame_loss_percent 0.000
min_latency_ns 12480.000
max_latency_ns 91560.000
avg_latency_ns 22684.000
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:33:45
guest_loopback_app testpmd testpmd
3.2.8.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 41184
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 6.04
majflt/s 0.00
VSZ 16454616
RSS 27557
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 41168
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52088
RSS 3456
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 41570
%usr 0.00
%system 0.00
%guest 200.10
%CPU 100.16
CPU
minflt/s 0.63
majflt/s 0.00
VSZ 2685324
RSS 73872
%MEM 0.11
kB_rd/s 0.00
kB_wr/s 8.09
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 41427
%usr 0.00
%system 0.01
%guest 200.13
%CPU 100.15
CPU
minflt/s 1.19
majflt/s 0.00
VSZ 2607496
RSS 101431
%MEM 0.15
kB_rd/s 0.00
kB_wr/s 0.18
kB_ccwr/s 0.00
3.2.8.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.8.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1424332.538
tx_rate_mbps Unknown
throughput_rx_mbps 17297.094
tx_rate_percent 87.625
throughput_rx_percent 87.625
frame_loss_percent 0.000
min_latency_ns 13160.000
max_latency_ns 76220.000
avg_latency_ns 18405.500
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:33:45
guest_loopback_app testpmd testpmd
3.2.8.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 41184
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 6.04
majflt/s 0.00
VSZ 16454616
RSS 27557
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 41168
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52088
RSS 3456
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 41570
%usr 0.00
%system 0.00
%guest 200.10
%CPU 100.16
CPU
minflt/s 0.63
majflt/s 0.00
VSZ 2685324
RSS 73872
%MEM 0.11
kB_rd/s 0.00
kB_wr/s 8.09
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 41427
%usr 0.00
%system 0.01
%guest 200.13
%CPU 100.15
CPU
minflt/s 1.19
majflt/s 0.00
VSZ 2607496
RSS 101431
%MEM 0.15
kB_rd/s 0.00
kB_wr/s 0.18
kB_ccwr/s 0.00
3.2.8.7. Anomalies

No anomalies were detected during the course of this test.

3.2.8.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.3. Rationale for decisions

The tests conducted do not have pass/fail/conditional-pass criteria. The test is simply conducted and the results are reported.

3.4. Conclusions and recommendations

The test results are stable. The vsperf CI jobs that were used to obtain the results can be found at https://artifacts.opnfv.org/logs/vswitchperf/intel-pod3/2016-12-01_21-05-27/vswitchperf_logs_2016-12-01_21-05-27.tar.gz.

3.5. General

3.5.1. Glossary

  • NFV - Network Function Virtualization
  • Mbps - 1,000,000bps

3.5.2. Document change procedures and history

Document ID Author Date Modified
OPNFV_vswitchperf_LTR_ver_1.0_Jan_15_CN_DRAFT Christopher Nolan 23/01/2015
OPNFV_vswitchperf_LTR_ver_1.1_Jan_15_CN_DRAFT Christopher Nolan 28/01/2015

4. Performance report for Open vSwitch

4.1. Introduction

The objective of the OPNFV project titled “Characterise vSwitch Performance for Telco NFV Use Cases”, is to evaluate a virtual switch to identify its suitability for a Telco Network Function Virtualization (NFV) environment. As well as this, the project aims to identify any gaps or bottlenecks in order to drive architectural changes to improve virtual switch performance and determinism. The purpose of this document is to summarize the results of the tests carried out on the virtual switch in the Network Function Virtualization Infrastructure (NFVI) and, from these results, provide evaluations and recommendations for the virtual switch. Test results will be outlined in details-of-LTR, preceded by the document-identifier and the scope and references).

This document is currently in draft form.

4.1.1. Document identifier

The document id will be used to uniquely identify versions of the LTR. The format for the document id will be: OPNFV_vswitchperf_LTR_rel_STATUS, the status is one of: DRAFT, REVIEWED, CORRECTED or FINAL. The document id for this version of the LTR is: OPNFV_vswitchperf_LTR_Brahmaputra_DRAFT.

4.1.2. Scope

The scope of this report is to detail the results of the tests that have been performed on the virtual switch. This report will also evaluate the results of these tests and, based on these evaluations, provide recommendations on the suitability of the virtual switch for use in a Telco NFV environment.

4.2. Details of the Level Test Report

This section provides a test-results-overview. Also included are the rationale and the conclusions.

4.2.1. Test ID: BACK2BACK

4.2.1.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 513153802b329c0584d72655460a4a264ad41e7c

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

Below are test details:

  • Test ID: back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: p2p
  • Traffic type: rfc2544
  • Bidirectional : True
4.2.1.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.1.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 5108
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:42:34
4.2.1.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 56972
%usr 0.35
%system 4.41
%guest 0.00
%CPU 4.76
CPU
minflt/s 5.73
majflt/s 0.00
VSZ 6609932
RSS 31570
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 56969
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.1.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 3382
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:42:34
4.2.1.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 56972
%usr 0.35
%system 4.41
%guest 0.00
%CPU 4.76
CPU
minflt/s 5.73
majflt/s 0.00
VSZ 6609932
RSS 31570
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 56969
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.1.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1344
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:42:34
4.2.1.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 56972
%usr 0.35
%system 4.41
%guest 0.00
%CPU 4.76
CPU
minflt/s 5.73
majflt/s 0.00
VSZ 6609932
RSS 31570
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 56969
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.1.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1096
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:42:34
4.2.1.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 56972
%usr 0.35
%system 4.41
%guest 0.00
%CPU 4.76
CPU
minflt/s 5.73
majflt/s 0.00
VSZ 6609932
RSS 31570
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 56969
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.1.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1116
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:42:34
4.2.1.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 56972
%usr 0.35
%system 4.41
%guest 0.00
%CPU 4.76
CPU
minflt/s 5.73
majflt/s 0.00
VSZ 6609932
RSS 31570
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 56969
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.7. Anomalies

No anomalies were detected during the course of this test.

4.2.1.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.2. Test ID: PVP_TPUT

4.2.2.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: 513153802b329c0584d72655460a4a264ad41e7c

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • VNF: QemuVirtioNet, Version: 2.5.0, GIT tag: a8c40fa2d667e585382080db36ac44e216b37a1c

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: pvp_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvp
  • Traffic type: rfc2544
  • Bidirectional : True
4.2.2.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.2.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 525542.895
tx_rate_mbps Unknown
throughput_rx_mbps 269.078
tx_rate_percent 1.773
throughput_rx_percent 1.766
frame_loss_percent 0.430
min_latency_ns 15060.000
max_latency_ns 30667100.000
avg_latency_ns 65954.500
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:29:58
guest_loopback_app testpmd
4.2.2.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 69554
%usr 0.35
%system 4.29
%guest 0.00
%CPU 4.64
CPU
minflt/s 5.78
majflt/s 0.00
VSZ 6609928
RSS 39357
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 69523
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 69839
%usr 0.00
%system 0.46
%guest 199.65
%CPU 100.16
CPU
minflt/s 0.46
majflt/s 0.00
VSZ 2660880
RSS 159743
%MEM 0.24
kB_rd/s 0.00
kB_wr/s 8.32
kB_ccwr/s 0.00
4.2.2.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.2.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 299546.481
tx_rate_mbps Unknown
throughput_rx_mbps 306.736
tx_rate_percent 1.773
throughput_rx_percent 1.773
frame_loss_percent 0.007
min_latency_ns 11840.000
max_latency_ns 2247820.000
avg_latency_ns 47069.500
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:29:58
guest_loopback_app testpmd
4.2.2.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 69554
%usr 0.35
%system 4.29
%guest 0.00
%CPU 4.64
CPU
minflt/s 5.78
majflt/s 0.00
VSZ 6609928
RSS 39357
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 69523
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 69839
%usr 0.00
%system 0.46
%guest 199.65
%CPU 100.16
CPU
minflt/s 0.46
majflt/s 0.00
VSZ 2660880
RSS 159743
%MEM 0.24
kB_rd/s 0.00
kB_wr/s 8.32
kB_ccwr/s 0.00
4.2.2.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.2.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 192375.600
tx_rate_mbps Unknown
throughput_rx_mbps 787.970
tx_rate_percent 4.094
throughput_rx_percent 4.094
frame_loss_percent 0.000
min_latency_ns 34700.000
max_latency_ns 298160.000
avg_latency_ns 106515.500
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:29:58
guest_loopback_app testpmd
4.2.2.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 69554
%usr 0.35
%system 4.29
%guest 0.00
%CPU 4.64
CPU
minflt/s 5.78
majflt/s 0.00
VSZ 6609928
RSS 39357
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 69523
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 69839
%usr 0.00
%system 0.46
%guest 199.65
%CPU 100.16
CPU
minflt/s 0.46
majflt/s 0.00
VSZ 2660880
RSS 159743
%MEM 0.24
kB_rd/s 0.00
kB_wr/s 8.32
kB_ccwr/s 0.00
4.2.2.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.2.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 190635.474
tx_rate_mbps Unknown
throughput_rx_mbps 1561.686
tx_rate_percent 7.961
throughput_rx_percent 7.961
frame_loss_percent 0.000
min_latency_ns 35640.000
max_latency_ns 247900.000
avg_latency_ns 97650.000
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:29:58
guest_loopback_app testpmd
4.2.2.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 69554
%usr 0.35
%system 4.29
%guest 0.00
%CPU 4.64
CPU
minflt/s 5.78
majflt/s 0.00
VSZ 6609928
RSS 39357
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 69523
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 69839
%usr 0.00
%system 0.46
%guest 199.65
%CPU 100.16
CPU
minflt/s 0.46
majflt/s 0.00
VSZ 2660880
RSS 159743
%MEM 0.24
kB_rd/s 0.00
kB_wr/s 8.32
kB_ccwr/s 0.00
4.2.2.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.2.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 129404.200
tx_rate_mbps Unknown
throughput_rx_mbps 1571.485
tx_rate_percent 7.961
throughput_rx_percent 7.961
frame_loss_percent 0.000
min_latency_ns 30680.000
max_latency_ns 271000.000
avg_latency_ns 93295.500
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:29:58
guest_loopback_app testpmd
4.2.2.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 69554
%usr 0.35
%system 4.29
%guest 0.00
%CPU 4.64
CPU
minflt/s 5.78
majflt/s 0.00
VSZ 6609928
RSS 39357
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 69523
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 69839
%usr 0.00
%system 0.46
%guest 199.65
%CPU 100.16
CPU
minflt/s 0.46
majflt/s 0.00
VSZ 2660880
RSS 159743
%MEM 0.24
kB_rd/s 0.00
kB_wr/s 8.32
kB_ccwr/s 0.00
4.2.2.7. Anomalies

No anomalies were detected during the course of this test.

4.2.2.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.3. Test ID: PHY2PHY_TPUT_MOD_VLAN

4.2.3.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 513153802b329c0584d72655460a4a264ad41e7c

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

Below are test details:

  • Test ID: phy2phy_tput_mod_vlan
  • Description: LTD.Throughput.RFC2544.PacketLossRatioFrameModification
  • Deployment: p2p
  • Traffic type: rfc2544
  • Bidirectional : False
4.2.3.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.3.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 609189.801
tx_rate_mbps Unknown
throughput_rx_mbps 331.399
tx_rate_percent 4.094
throughput_rx_percent 4.289
frame_loss_percent 0.000
min_latency_ns 4160.000
max_latency_ns 1643960.000
avg_latency_ns 55386.000
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:28:14
4.2.3.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 62217
%usr 0.36
%system 4.49
%guest 0.00
%CPU 4.85
CPU
minflt/s 5.70
majflt/s 0.00
VSZ 6609928
RSS 30853
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 62214
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45897
RSS 3255
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.3.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.3.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 672377.342
tx_rate_mbps Unknown
throughput_rx_mbps 710.030
tx_rate_percent 7.961
throughput_rx_percent 8.176
frame_loss_percent 0.000
min_latency_ns 4400.000
max_latency_ns 3519060.000
avg_latency_ns 201152.000
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:28:14
4.2.3.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 62217
%usr 0.36
%system 4.49
%guest 0.00
%CPU 4.85
CPU
minflt/s 5.70
majflt/s 0.00
VSZ 6609928
RSS 30853
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 62214
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45897
RSS 3255
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.3.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.3.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 586856.647
tx_rate_mbps Unknown
throughput_rx_mbps 2422.544
tx_rate_percent 24.977
throughput_rx_percent 25.164
frame_loss_percent 0.000
min_latency_ns 5320.000
max_latency_ns 2153580.000
avg_latency_ns 131866.000
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:28:14
4.2.3.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 62217
%usr 0.36
%system 4.49
%guest 0.00
%CPU 4.85
CPU
minflt/s 5.70
majflt/s 0.00
VSZ 6609928
RSS 30853
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 62214
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45897
RSS 3255
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.3.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.3.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 576863.978
tx_rate_mbps Unknown
throughput_rx_mbps 4744.129
tx_rate_percent 48.180
throughput_rx_percent 48.364
frame_loss_percent 0.000
min_latency_ns 5820.000
max_latency_ns 1491820.000
avg_latency_ns 100731.000
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:28:14
4.2.3.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 62217
%usr 0.36
%system 4.49
%guest 0.00
%CPU 4.85
CPU
minflt/s 5.70
majflt/s 0.00
VSZ 6609928
RSS 30853
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 62214
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45897
RSS 3255
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.3.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.3.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 561302.291
tx_rate_mbps Unknown
throughput_rx_mbps 6834.417
tx_rate_percent 69.063
throughput_rx_percent 69.242
frame_loss_percent 0.000
min_latency_ns 6400.000
max_latency_ns 910960.000
avg_latency_ns 79007.000
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:28:14
4.2.3.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 62217
%usr 0.36
%system 4.49
%guest 0.00
%CPU 4.85
CPU
minflt/s 5.70
majflt/s 0.00
VSZ 6609928
RSS 30853
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 62214
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45897
RSS 3255
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.3.7. Anomalies

No anomalies were detected during the course of this test.

4.2.3.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.4. Test ID: PVVP_BACK2BACK

4.2.4.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: pvvp

  • vswitchperf: GIT tag: 513153802b329c0584d72655460a4a264ad41e7c

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • VNF: QemuVirtioNet, Version: 2.5.0, GIT tag: a8c40fa2d667e585382080db36ac44e216b37a1c

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: pvvp_back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: pvvp
  • Traffic type: rfc2544
  • Bidirectional : True
4.2.4.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.4.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:43:49
guest_loopback_app testpmd testpmd
4.2.4.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 83226
%usr 0.34
%system 4.26
%guest 0.00
%CPU 4.60
CPU
minflt/s 6.66
majflt/s 0.00
VSZ 6610056
RSS 31314
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 83223
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 83740
%usr 0.00
%system 0.03
%guest 200.08
%CPU 100.16
CPU
minflt/s 0.33
majflt/s 0.00
VSZ 2612760
RSS 145134
%MEM 0.22
kB_rd/s 0.00
kB_wr/s 5.73
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 83585
%usr 0.00
%system 0.02
%guest 200.08
%CPU 100.16
CPU
minflt/s 0.06
majflt/s 0.00
VSZ 2719260
RSS 179823
%MEM 0.27
kB_rd/s 0.00
kB_wr/s 0.05
kB_ccwr/s 0.00
4.2.4.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.4.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:43:49
guest_loopback_app testpmd testpmd
4.2.4.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 83226
%usr 0.34
%system 4.26
%guest 0.00
%CPU 4.60
CPU
minflt/s 6.66
majflt/s 0.00
VSZ 6610056
RSS 31314
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 83223
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 83740
%usr 0.00
%system 0.03
%guest 200.08
%CPU 100.16
CPU
minflt/s 0.33
majflt/s 0.00
VSZ 2612760
RSS 145134
%MEM 0.22
kB_rd/s 0.00
kB_wr/s 5.73
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 83585
%usr 0.00
%system 0.02
%guest 200.08
%CPU 100.16
CPU
minflt/s 0.06
majflt/s 0.00
VSZ 2719260
RSS 179823
%MEM 0.27
kB_rd/s 0.00
kB_wr/s 0.05
kB_ccwr/s 0.00
4.2.4.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.4.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:43:49
guest_loopback_app testpmd testpmd
4.2.4.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 83226
%usr 0.34
%system 4.26
%guest 0.00
%CPU 4.60
CPU
minflt/s 6.66
majflt/s 0.00
VSZ 6610056
RSS 31314
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 83223
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 83740
%usr 0.00
%system 0.03
%guest 200.08
%CPU 100.16
CPU
minflt/s 0.33
majflt/s 0.00
VSZ 2612760
RSS 145134
%MEM 0.22
kB_rd/s 0.00
kB_wr/s 5.73
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 83585
%usr 0.00
%system 0.02
%guest 200.08
%CPU 100.16
CPU
minflt/s 0.06
majflt/s 0.00
VSZ 2719260
RSS 179823
%MEM 0.27
kB_rd/s 0.00
kB_wr/s 0.05
kB_ccwr/s 0.00
4.2.4.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.4.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:43:49
guest_loopback_app testpmd testpmd
4.2.4.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 83226
%usr 0.34
%system 4.26
%guest 0.00
%CPU 4.60
CPU
minflt/s 6.66
majflt/s 0.00
VSZ 6610056
RSS 31314
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 83223
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 83740
%usr 0.00
%system 0.03
%guest 200.08
%CPU 100.16
CPU
minflt/s 0.33
majflt/s 0.00
VSZ 2612760
RSS 145134
%MEM 0.22
kB_rd/s 0.00
kB_wr/s 5.73
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 83585
%usr 0.00
%system 0.02
%guest 200.08
%CPU 100.16
CPU
minflt/s 0.06
majflt/s 0.00
VSZ 2719260
RSS 179823
%MEM 0.27
kB_rd/s 0.00
kB_wr/s 0.05
kB_ccwr/s 0.00
4.2.4.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.4.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:43:49
guest_loopback_app testpmd testpmd
4.2.4.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 83226
%usr 0.34
%system 4.26
%guest 0.00
%CPU 4.60
CPU
minflt/s 6.66
majflt/s 0.00
VSZ 6610056
RSS 31314
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 83223
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 83740
%usr 0.00
%system 0.03
%guest 200.08
%CPU 100.16
CPU
minflt/s 0.33
majflt/s 0.00
VSZ 2612760
RSS 145134
%MEM 0.22
kB_rd/s 0.00
kB_wr/s 5.73
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 83585
%usr 0.00
%system 0.02
%guest 200.08
%CPU 100.16
CPU
minflt/s 0.06
majflt/s 0.00
VSZ 2719260
RSS 179823
%MEM 0.27
kB_rd/s 0.00
kB_wr/s 0.05
kB_ccwr/s 0.00
4.2.4.7. Anomalies

No anomalies were detected during the course of this test.

4.2.4.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.5. Test ID: PHY2PHY_TPUT

4.2.5.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 513153802b329c0584d72655460a4a264ad41e7c

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

Below are test details:

  • Test ID: phy2phy_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: p2p
  • Traffic type: rfc2544
  • Bidirectional : True
4.2.5.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.5.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 758000.041
tx_rate_mbps Unknown
throughput_rx_mbps 388.096
tx_rate_percent 2.547
throughput_rx_percent 2.547
frame_loss_percent 0.000
min_latency_ns 4020.000
max_latency_ns 6907040.000
avg_latency_ns 715024.500
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:28:01
4.2.5.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 53342
%usr 0.36
%system 4.48
%guest 0.00
%CPU 4.84
CPU
minflt/s 5.46
majflt/s 0.00
VSZ 6609932
RSS 31064
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 53339
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45916
RSS 3232
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.5.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.5.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 691513.693
tx_rate_mbps Unknown
throughput_rx_mbps 708.110
tx_rate_percent 4.094
throughput_rx_percent 4.094
frame_loss_percent 0.000
min_latency_ns 4360.000
max_latency_ns 3816640.000
avg_latency_ns 208523.000
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:28:01
4.2.5.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 53342
%usr 0.36
%system 4.48
%guest 0.00
%CPU 4.84
CPU
minflt/s 5.46
majflt/s 0.00
VSZ 6609932
RSS 31064
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 53339
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45916
RSS 3232
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.5.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.5.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 628537.394
tx_rate_mbps Unknown
throughput_rx_mbps 2574.489
tx_rate_percent 13.375
throughput_rx_percent 13.375
frame_loss_percent 0.000
min_latency_ns 5220.000
max_latency_ns 3758940.000
avg_latency_ns 248198.500
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:28:01
4.2.5.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 53342
%usr 0.36
%system 4.48
%guest 0.00
%CPU 4.84
CPU
minflt/s 5.46
majflt/s 0.00
VSZ 6609932
RSS 31064
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 53339
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45916
RSS 3232
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.5.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.5.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 579578.667
tx_rate_mbps Unknown
throughput_rx_mbps 4747.908
tx_rate_percent 24.203
throughput_rx_percent 24.203
frame_loss_percent 0.000
min_latency_ns 5860.000
max_latency_ns 2119040.000
avg_latency_ns 91851.500
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:28:01
4.2.5.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 53342
%usr 0.36
%system 4.48
%guest 0.00
%CPU 4.84
CPU
minflt/s 5.46
majflt/s 0.00
VSZ 6609932
RSS 31064
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 53339
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45916
RSS 3232
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.5.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.5.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 556857.917
tx_rate_mbps Unknown
throughput_rx_mbps 6762.483
tx_rate_percent 34.258
throughput_rx_percent 34.258
frame_loss_percent 0.000
min_latency_ns 6480.000
max_latency_ns 1178060.000
avg_latency_ns 59141.000
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:28:01
4.2.5.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 53342
%usr 0.36
%system 4.48
%guest 0.00
%CPU 4.84
CPU
minflt/s 5.46
majflt/s 0.00
VSZ 6609932
RSS 31064
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 53339
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45916
RSS 3232
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.5.7. Anomalies

No anomalies were detected during the course of this test.

4.2.5.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.6. Test ID: PHY2PHY_SCALABILITY

4.2.6.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 513153802b329c0584d72655460a4a264ad41e7c

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

Below are test details:

  • Test ID: phy2phy_scalability
  • Description: LTD.Scalability.Flows.RFC2544.0PacketLoss
  • Deployment: p2p
  • Traffic type: rfc2544
  • Bidirectional : True
4.2.6.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.6.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 758000.722
tx_rate_mbps Unknown
throughput_rx_mbps 388.096
tx_rate_percent 2.547
throughput_rx_percent 2.547
frame_loss_percent 0.000
min_latency_ns 4020.000
max_latency_ns 7733040.000
avg_latency_ns 651109.500
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:28:56
stream_count 8000
match_type L4
pre-installed_flows No
4.2.6.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 65907
%usr 0.35
%system 4.47
%guest 0.00
%CPU 4.82
CPU
minflt/s 6.19
majflt/s 0.00
VSZ 6609932
RSS 38487
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 65904
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.6.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.6.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 691513.534
tx_rate_mbps Unknown
throughput_rx_mbps 708.110
tx_rate_percent 4.094
throughput_rx_percent 4.094
frame_loss_percent 0.000
min_latency_ns 4300.000
max_latency_ns 4220780.000
avg_latency_ns 232968.500
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:28:56
stream_count 8000
match_type L4
pre-installed_flows No
4.2.6.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 65907
%usr 0.35
%system 4.47
%guest 0.00
%CPU 4.82
CPU
minflt/s 6.19
majflt/s 0.00
VSZ 6609932
RSS 38487
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 65904
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.6.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.6.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 628538.249
tx_rate_mbps Unknown
throughput_rx_mbps 2574.493
tx_rate_percent 13.375
throughput_rx_percent 13.375
frame_loss_percent 0.000
min_latency_ns 5240.000
max_latency_ns 3977340.000
avg_latency_ns 291694.500
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:28:56
stream_count 8000
match_type L4
pre-installed_flows No
4.2.6.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 65907
%usr 0.35
%system 4.47
%guest 0.00
%CPU 4.82
CPU
minflt/s 6.19
majflt/s 0.00
VSZ 6609932
RSS 38487
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 65904
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.6.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.6.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 579578.553
tx_rate_mbps Unknown
throughput_rx_mbps 4747.908
tx_rate_percent 24.203
throughput_rx_percent 24.203
frame_loss_percent 0.000
min_latency_ns 5820.000
max_latency_ns 1508820.000
avg_latency_ns 84438.000
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:28:56
stream_count 8000
match_type L4
pre-installed_flows No
4.2.6.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 65907
%usr 0.35
%system 4.47
%guest 0.00
%CPU 4.82
CPU
minflt/s 6.19
majflt/s 0.00
VSZ 6609932
RSS 38487
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 65904
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.6.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.6.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 594574.988
tx_rate_mbps Unknown
throughput_rx_mbps 7220.519
tx_rate_percent 36.578
throughput_rx_percent 36.578
frame_loss_percent 0.000
min_latency_ns 6520.000
max_latency_ns 2374480.000
avg_latency_ns 113618.000
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:28:56
stream_count 8000
match_type L4
pre-installed_flows No
4.2.6.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 65907
%usr 0.35
%system 4.47
%guest 0.00
%CPU 4.82
CPU
minflt/s 6.19
majflt/s 0.00
VSZ 6609932
RSS 38487
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 65904
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.6.7. Anomalies

No anomalies were detected during the course of this test.

4.2.6.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.7. Test ID: PVP_BACK2BACK

4.2.7.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: 513153802b329c0584d72655460a4a264ad41e7c

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • VNF: QemuVirtioNet, Version: 2.5.0, GIT tag: a8c40fa2d667e585382080db36ac44e216b37a1c

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: pvp_back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: pvp
  • Traffic type: rfc2544
  • Bidirectional : True
4.2.7.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.7.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:43:05
guest_loopback_app testpmd
4.2.7.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 73559
%usr 0.35
%system 4.36
%guest 0.00
%CPU 4.71
CPU
minflt/s 5.89
majflt/s 0.00
VSZ 6609932
RSS 32018
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 73556
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3264
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 73851
%usr 0.00
%system 0.03
%guest 200.07
%CPU 100.15
CPU
minflt/s 0.24
majflt/s 0.00
VSZ 2685452
RSS 247525
%MEM 0.38
kB_rd/s 0.00
kB_wr/s 5.01
kB_ccwr/s 0.00
4.2.7.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.7.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:43:05
guest_loopback_app testpmd
4.2.7.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 73559
%usr 0.35
%system 4.36
%guest 0.00
%CPU 4.71
CPU
minflt/s 5.89
majflt/s 0.00
VSZ 6609932
RSS 32018
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 73556
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3264
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 73851
%usr 0.00
%system 0.03
%guest 200.07
%CPU 100.15
CPU
minflt/s 0.24
majflt/s 0.00
VSZ 2685452
RSS 247525
%MEM 0.38
kB_rd/s 0.00
kB_wr/s 5.01
kB_ccwr/s 0.00
4.2.7.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.7.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:43:05
guest_loopback_app testpmd
4.2.7.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 73559
%usr 0.35
%system 4.36
%guest 0.00
%CPU 4.71
CPU
minflt/s 5.89
majflt/s 0.00
VSZ 6609932
RSS 32018
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 73556
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3264
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 73851
%usr 0.00
%system 0.03
%guest 200.07
%CPU 100.15
CPU
minflt/s 0.24
majflt/s 0.00
VSZ 2685452
RSS 247525
%MEM 0.38
kB_rd/s 0.00
kB_wr/s 5.01
kB_ccwr/s 0.00
4.2.7.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.7.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:43:05
guest_loopback_app testpmd
4.2.7.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 73559
%usr 0.35
%system 4.36
%guest 0.00
%CPU 4.71
CPU
minflt/s 5.89
majflt/s 0.00
VSZ 6609932
RSS 32018
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 73556
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3264
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 73851
%usr 0.00
%system 0.03
%guest 200.07
%CPU 100.15
CPU
minflt/s 0.24
majflt/s 0.00
VSZ 2685452
RSS 247525
%MEM 0.38
kB_rd/s 0.00
kB_wr/s 5.01
kB_ccwr/s 0.00
4.2.7.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.7.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:43:05
guest_loopback_app testpmd
4.2.7.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 73559
%usr 0.35
%system 4.36
%guest 0.00
%CPU 4.71
CPU
minflt/s 5.89
majflt/s 0.00
VSZ 6609932
RSS 32018
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 73556
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3264
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 73851
%usr 0.00
%system 0.03
%guest 200.07
%CPU 100.15
CPU
minflt/s 0.24
majflt/s 0.00
VSZ 2685452
RSS 247525
%MEM 0.38
kB_rd/s 0.00
kB_wr/s 5.01
kB_ccwr/s 0.00
4.2.7.7. Anomalies

No anomalies were detected during the course of this test.

4.2.7.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.8. Test ID: PVVP_TPUT

4.2.8.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: pvvp

  • vswitchperf: GIT tag: 513153802b329c0584d72655460a4a264ad41e7c

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • VNF: QemuVirtioNet, Version: 2.5.0, GIT tag: a8c40fa2d667e585382080db36ac44e216b37a1c

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: pvvp_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvvp
  • Traffic type: rfc2544
  • Bidirectional : True
4.2.8.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.8.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 518378.129
tx_rate_mbps Unknown
throughput_rx_mbps 265.410
tx_rate_percent 1.773
throughput_rx_percent 1.742
frame_loss_percent 1.793
min_latency_ns 25660.000
max_latency_ns 31289860.000
avg_latency_ns 217831.000
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:30:05
guest_loopback_app testpmd testpmd
4.2.8.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 79088
%usr 0.34
%system 4.30
%guest 0.00
%CPU 4.65
CPU
minflt/s 7.29
majflt/s 0.00
VSZ 6610056
RSS 35289
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 79085
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3272
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 79586
%usr 0.00
%system 0.92
%guest 199.68
%CPU 100.17
CPU
minflt/s 1.67
majflt/s 0.00
VSZ 2578968
RSS 63015
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.32
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 79447
%usr 0.00
%system 0.94
%guest 199.52
%CPU 100.17
CPU
minflt/s 1.06
majflt/s 0.00
VSZ 2734612
RSS 99926
%MEM 0.15
kB_rd/s 0.00
kB_wr/s 0.08
kB_ccwr/s 0.00
4.2.8.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.8.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 299551.359
tx_rate_mbps Unknown
throughput_rx_mbps 306.741
tx_rate_percent 1.773
throughput_rx_percent 1.773
frame_loss_percent 0.006
min_latency_ns 18520.000
max_latency_ns 2178040.000
avg_latency_ns 58897.500
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:30:05
guest_loopback_app testpmd testpmd
4.2.8.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 79088
%usr 0.34
%system 4.30
%guest 0.00
%CPU 4.65
CPU
minflt/s 7.29
majflt/s 0.00
VSZ 6610056
RSS 35289
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 79085
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3272
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 79586
%usr 0.00
%system 0.92
%guest 199.68
%CPU 100.17
CPU
minflt/s 1.67
majflt/s 0.00
VSZ 2578968
RSS 63015
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.32
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 79447
%usr 0.00
%system 0.94
%guest 199.52
%CPU 100.17
CPU
minflt/s 1.06
majflt/s 0.00
VSZ 2734612
RSS 99926
%MEM 0.15
kB_rd/s 0.00
kB_wr/s 0.08
kB_ccwr/s 0.00
4.2.8.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.8.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 265068.231
tx_rate_mbps Unknown
throughput_rx_mbps 1085.719
tx_rate_percent 5.641
throughput_rx_percent 5.641
frame_loss_percent 0.000
min_latency_ns 47080.000
max_latency_ns 763980.000
avg_latency_ns 131176.500
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:30:05
guest_loopback_app testpmd testpmd
4.2.8.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 79088
%usr 0.34
%system 4.30
%guest 0.00
%CPU 4.65
CPU
minflt/s 7.29
majflt/s 0.00
VSZ 6610056
RSS 35289
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 79085
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3272
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 79586
%usr 0.00
%system 0.92
%guest 199.68
%CPU 100.17
CPU
minflt/s 1.67
majflt/s 0.00
VSZ 2578968
RSS 63015
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.32
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 79447
%usr 0.00
%system 0.94
%guest 199.52
%CPU 100.17
CPU
minflt/s 1.06
majflt/s 0.00
VSZ 2734612
RSS 99926
%MEM 0.15
kB_rd/s 0.00
kB_wr/s 0.08
kB_ccwr/s 0.00
4.2.8.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.8.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 246199.544
tx_rate_mbps Unknown
throughput_rx_mbps 2016.867
tx_rate_percent 10.281
throughput_rx_percent 10.281
frame_loss_percent 0.000
min_latency_ns 55780.000
max_latency_ns 591020.000
avg_latency_ns 135571.000
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:30:05
guest_loopback_app testpmd testpmd
4.2.8.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 79088
%usr 0.34
%system 4.30
%guest 0.00
%CPU 4.65
CPU
minflt/s 7.29
majflt/s 0.00
VSZ 6610056
RSS 35289
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 79085
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3272
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 79586
%usr 0.00
%system 0.92
%guest 199.68
%CPU 100.17
CPU
minflt/s 1.67
majflt/s 0.00
VSZ 2578968
RSS 63015
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.32
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 79447
%usr 0.00
%system 0.94
%guest 199.52
%CPU 100.17
CPU
minflt/s 1.06
majflt/s 0.00
VSZ 2734612
RSS 99926
%MEM 0.15
kB_rd/s 0.00
kB_wr/s 0.08
kB_ccwr/s 0.00
4.2.8.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.8.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 217420.578
tx_rate_mbps Unknown
throughput_rx_mbps 2640.355
tx_rate_percent 13.375
throughput_rx_percent 13.375
frame_loss_percent 0.000
min_latency_ns 64300.000
max_latency_ns 529340.000
avg_latency_ns 161071.000
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:30:05
guest_loopback_app testpmd testpmd
4.2.8.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 79088
%usr 0.34
%system 4.30
%guest 0.00
%CPU 4.65
CPU
minflt/s 7.29
majflt/s 0.00
VSZ 6610056
RSS 35289
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 79085
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3272
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 79586
%usr 0.00
%system 0.92
%guest 199.68
%CPU 100.17
CPU
minflt/s 1.67
majflt/s 0.00
VSZ 2578968
RSS 63015
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.32
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 79447
%usr 0.00
%system 0.94
%guest 199.52
%CPU 100.17
CPU
minflt/s 1.06
majflt/s 0.00
VSZ 2734612
RSS 99926
%MEM 0.15
kB_rd/s 0.00
kB_wr/s 0.08
kB_ccwr/s 0.00
4.2.8.7. Anomalies

No anomalies were detected during the course of this test.

4.2.8.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.3. Rationale for decisions

The tests conducted do not have pass/fail/conditional-pass criteria. The test is simply conducted and the results are reported.

4.4. Conclusions and recommendations

The test results are stable. The vsperf CI jobs that were used to obtain the results can be found at https://artifacts.opnfv.org/logs/vswitchperf/intel-pod3/2016-12-01_21-05-27/vswitchperf_logs_2016-12-01_21-05-27.tar.gz.

4.5. General

4.5.1. Glossary

  • NFV - Network Function Virtualization
  • Mbps - 1,000,000bps

4.5.2. Document change procedures and history

Document ID Author Date Modified
OPNFV_vswitchperf_LTR_ver_1.0_Jan_15_CN_DRAFT Christopher Nolan 23/01/2015
OPNFV_vswitchperf_LTR_ver_1.1_Jan_15_CN_DRAFT Christopher Nolan 28/01/2015