VSPERF Results

1. OPNFV Brahmaputra Scenarios

Available Tests and aspects of scenarios:

Framework Test Definition
phy2phy_tput PacketLossRatio for Phy2Phy
back2back BackToBackFrames for Phy2Phy
phy2phy_tput_mod_vlan PacketLossRatioFrameModification for Phy2Phy
phy2phy_cont Phy2Phy blast vswitch at x% TX rate and measure throughput
pvp_cont PVP blast vswitch at x% TX rate and measure throughput
pvvp_cont PVVP blast vswitch at x% TX rate and measure throughput
phy2phy_scalability Scalability0PacketLoss for Phy2Phy
pvp_tput PacketLossRatio for PVP
pvp_back2back BackToBackFrames for PVP
pvvp_tput PacketLossRatio for PVVP
pvvp_back2back BackToBackFrames for PVVP
phy2phy_cpu_load CPU0PacketLoss for Phy2Phy
phy2phy_mem_load Same as CPU0PacketLoss but using a memory intensive app

Supported deployment scenarios:

  • Phy2Phy: Physical port -> vSwitch -> Physical port.
  • PVP: Physical port -> vSwitch -> VNF -> vSwitch -> Physical port.
  • PVVP: Physical port -> vSwitch -> VNF -> vSwitch -> VNF -> vSwitch -> Physical port.

Loopback applications in the Guest can be:

Supported traffic generators:

  • Ixia: IxOS and IxNet.
  • Spirent.
  • Dummy.

2. OPNFV Brahmaputra Results

The vsperf CI jobs that were used to obtain the results can be found at https://wiki.opnfv.org/wiki/vsperf_results.

The following table maps the results in the test dashboard to the appropriate test case in the VSPERF Framework and specifies the metric the vertical/Y axis is plotting. Please note, the presence of dpdk within a test name signifies that the vswitch under test was OVS with DPDK, while its absence indicates that the vswitch under test was stock OVS.

Dashboard Test Framework Test Metric Guest Interface
tput_ovsdpdk phy2phy_tput Throughput (FPS) N/A
tput_ovs phy2phy_tput Throughput (FPS) N/A
b2b_ovsdpdk back2back Back-to-back value N/A
b2b_ovs back2back Back-to-back value N/A
tput_mod_vlan_ovs phy2phy_tput_mod_vlan Throughput (FPS) N/A
tput_mod_vlan_ovsdpdk phy2phy_tput_mod_vlan Throughput (FPS) N/A
scalability_ovs phy2phy_scalability Throughput (FPS) N/A
scalability_ovsdpdk phy2phy_scalability Throughput (FPS) N/A
pvp_tput_ovsdpdkuser pvp_tput Throughput (FPS) vhost-user
pvp_tput_ovsvirtio pvp_tput Throughput (FPS) virtio-net
pvp_b2b_ovsdpdkuser pvp_back2back Back-to-back value vhost-user
pvp_b2b_ovsvirtio pvp_back2back Back-to-back value virtio-net
pvvp_tput_ovsdpdkuser pvvp_tput Throughput (FPS) vhost-user
pvvp_tput_ovsvirtio pvvp_tput Throughput (FPS) virtio-net
pvvp_b2b_ovsdpdkuser pvvp_back2back Throughput (FPS) vhost-user
pvvp_b2b_ovsvirtio pvvp_back2back Throughput (FPS) virtio-net

The loopback application in the VNF used for PVP and PVVP scenarios was DPDK testpmd.

3. Performance report for Open vSwitch with DPDK support

3.1. Introduction

The objective of the OPNFV project titled “Characterise vSwitch Performance for Telco NFV Use Cases”, is to evaluate a virtual switch to identify its suitability for a Telco Network Function Virtualization (NFV) environment. As well as this, the project aims to identify any gaps or bottlenecks in order to drive architectural changes to improve virtual switch performance and determinism. The purpose of this document is to summarize the results of the tests carried out on the virtual switch in the Network Function Virtualization Infrastructure (NFVI) and, from these results, provide evaluations and recommendations for the virtual switch. Test results will be outlined in details-of-LTR, preceded by the document-identifier and the scope and references).

This document is currently in draft form.

3.1.1. Document identifier

The document id will be used to uniquely identify versions of the LTR. The format for the document id will be: OPNFV_vswitchperf_LTR_rel_STATUS, the status is one of: DRAFT, REVIEWED, CORRECTED or FINAL. The document id for this version of the LTR is: OPNFV_vswitchperf_LTR_Brahmaputra_DRAFT.

3.1.2. Scope

The scope of this report is to detail the results of the tests that have been performed on the virtual switch. This report will also evaluate the results of these tests and, based on these evaluations, provide recommendations on the suitability of the virtual switch for use in a Telco NFV environment.

3.2. Details of the Level Test Report

This section provides a test-results-overview. Also included are the rationale and the conclusions.

3.2.1. Test ID: BACK2BACK

3.2.1.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: d3607cb18d6d3f55b53c06479c748b0184ba0ac5

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • DPDK Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: p2p
  • Traffic type: rfc2544
  • Bidirectional : True
3.2.1.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.1.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 27247
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:44:01
3.2.1.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 1534
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 19.41
majflt/s 0.00
VSZ 8057816
RSS 25362
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 1518
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52080
RSS 3480
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.1.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 39590371
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:44:01
3.2.1.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 1534
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 19.41
majflt/s 0.00
VSZ 8057816
RSS 25362
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 1518
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52080
RSS 3480
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.1.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 70488721
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:44:01
3.2.1.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 1534
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 19.41
majflt/s 0.00
VSZ 8057816
RSS 25362
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 1518
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52080
RSS 3480
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.1.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 35919540
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:44:01
3.2.1.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 1534
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 19.41
majflt/s 0.00
VSZ 8057816
RSS 25362
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 1518
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52080
RSS 3480
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.1.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 24382314
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:44:01
3.2.1.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 1534
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 19.41
majflt/s 0.00
VSZ 8057816
RSS 25362
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 1518
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52080
RSS 3480
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.7. Anomalies

No anomalies were detected during the course of this test.

3.2.1.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.2. Test ID: PVP_TPUT

3.2.2.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: d3607cb18d6d3f55b53c06479c748b0184ba0ac5

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • DPDK Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

  • VNF: QemuDpdkVhostUser, Version: 2.5.0, GIT tag: a8c40fa2d667e585382080db36ac44e216b37a1c

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: pvp_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvp
  • Traffic type: rfc2544
  • Bidirectional : True
3.2.2.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.2.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4441037.546
tx_rate_mbps Unknown
throughput_rx_mbps 2273.811
tx_rate_percent 14.922
throughput_rx_percent 14.922
frame_loss_percent 0.000
min_latency_ns 4760.000
max_latency_ns 97780.000
avg_latency_ns 6384.500
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:38:41
guest_loopback_app testpmd
3.2.2.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 17150
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 14.40
majflt/s 0.00
VSZ 12256220
RSS 25431
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 17134
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52080
RSS 3488
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 17381
%usr 0.00
%system 0.19
%guest 200.21
%CPU 100.16
CPU
minflt/s 1.55
majflt/s 0.00
VSZ 2647436
RSS 151888
%MEM 0.23
kB_rd/s 0.00
kB_wr/s 6.33
kB_ccwr/s 0.00
3.2.2.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.2.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 5394858.148
tx_rate_mbps Unknown
throughput_rx_mbps 5524.335
tx_rate_percent 31.938
throughput_rx_percent 31.938
frame_loss_percent 0.000
min_latency_ns 6520.000
max_latency_ns 101740.000
avg_latency_ns 9094.500
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:38:41
guest_loopback_app testpmd
3.2.2.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 17150
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 14.40
majflt/s 0.00
VSZ 12256220
RSS 25431
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 17134
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52080
RSS 3488
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 17381
%usr 0.00
%system 0.19
%guest 200.21
%CPU 100.16
CPU
minflt/s 1.55
majflt/s 0.00
VSZ 2647436
RSS 151888
%MEM 0.23
kB_rd/s 0.00
kB_wr/s 6.33
kB_ccwr/s 0.00
3.2.2.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.2.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699093.720
tx_rate_mbps Unknown
throughput_rx_mbps 19247.488
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 44560.000
max_latency_ns 156440.000
avg_latency_ns 138867.500
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:38:41
guest_loopback_app testpmd
3.2.2.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 17150
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 14.40
majflt/s 0.00
VSZ 12256220
RSS 25431
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 17134
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52080
RSS 3488
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 17381
%usr 0.00
%system 0.19
%guest 200.21
%CPU 100.16
CPU
minflt/s 1.55
majflt/s 0.00
VSZ 2647436
RSS 151888
%MEM 0.23
kB_rd/s 0.00
kB_wr/s 6.33
kB_ccwr/s 0.00
3.2.2.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.2.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394586.739
tx_rate_mbps Unknown
throughput_rx_mbps 19616.455
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 65820.000
max_latency_ns 129580.000
avg_latency_ns 116977.000
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:38:41
guest_loopback_app testpmd
3.2.2.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 17150
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 14.40
majflt/s 0.00
VSZ 12256220
RSS 25431
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 17134
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52080
RSS 3488
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 17381
%usr 0.00
%system 0.19
%guest 200.21
%CPU 100.16
CPU
minflt/s 1.55
majflt/s 0.00
VSZ 2647436
RSS 151888
%MEM 0.23
kB_rd/s 0.00
kB_wr/s 6.33
kB_ccwr/s 0.00
3.2.2.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.2.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625453.486
tx_rate_mbps Unknown
throughput_rx_mbps 19739.507
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 52380.000
max_latency_ns 133140.000
avg_latency_ns 114715.000
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:38:41
guest_loopback_app testpmd
3.2.2.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 17150
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 14.40
majflt/s 0.00
VSZ 12256220
RSS 25431
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 17134
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52080
RSS 3488
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 17381
%usr 0.00
%system 0.19
%guest 200.21
%CPU 100.16
CPU
minflt/s 1.55
majflt/s 0.00
VSZ 2647436
RSS 151888
%MEM 0.23
kB_rd/s 0.00
kB_wr/s 6.33
kB_ccwr/s 0.00
3.2.2.7. Anomalies

No anomalies were detected during the course of this test.

3.2.2.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.3. Test ID: PHY2PHY_TPUT_MOD_VLAN

3.2.3.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: d3607cb18d6d3f55b53c06479c748b0184ba0ac5

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • DPDK Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: phy2phy_tput_mod_vlan
  • Description: LTD.Throughput.RFC2544.PacketLossRatioFrameModification
  • Deployment: p2p
  • Traffic type: rfc2544
  • Bidirectional : False
3.2.3.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.3.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 10047017.679
tx_rate_mbps Unknown
throughput_rx_mbps 5465.578
tx_rate_percent 67.516
throughput_rx_percent 70.731
frame_loss_percent 0.000
min_latency_ns 4940.000
max_latency_ns 422060.000
avg_latency_ns 7171.000
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:40:58
3.2.3.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 7268
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 16.51
majflt/s 0.00
VSZ 8057816
RSS 25328
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 7252
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52084
RSS 3480
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.3.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.3.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 7531450.198
tx_rate_mbps Unknown
throughput_rx_mbps 7953.211
tx_rate_percent 89.172
throughput_rx_percent 91.582
frame_loss_percent 0.000
min_latency_ns 4680.000
max_latency_ns 94240.000
avg_latency_ns 5433.000
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:40:58
3.2.3.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 7268
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 16.51
majflt/s 0.00
VSZ 8057816
RSS 25328
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 7252
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52084
RSS 3480
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.3.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.3.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2331451.575
tx_rate_mbps Unknown
throughput_rx_mbps 9624.232
tx_rate_percent 99.227
throughput_rx_percent 99.973
frame_loss_percent 0.000
min_latency_ns 5980.000
max_latency_ns 33620.000
avg_latency_ns 13467.000
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:40:58
3.2.3.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 7268
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 16.51
majflt/s 0.00
VSZ 8057816
RSS 25328
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 7252
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52084
RSS 3480
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.3.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.3.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1188058.523
tx_rate_mbps Unknown
throughput_rx_mbps 9770.593
tx_rate_percent 99.227
throughput_rx_percent 99.607
frame_loss_percent 0.000
min_latency_ns 5760.000
max_latency_ns 366280.000
avg_latency_ns 8165.000
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:40:58
3.2.3.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 7268
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 16.51
majflt/s 0.00
VSZ 8057816
RSS 25328
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 7252
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52084
RSS 3480
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.3.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.3.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 806458.489
tx_rate_mbps Unknown
throughput_rx_mbps 9819.439
tx_rate_percent 99.227
throughput_rx_percent 99.485
frame_loss_percent 0.000
min_latency_ns 6220.000
max_latency_ns 34580.000
avg_latency_ns 7545.000
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:40:58
3.2.3.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 7268
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 16.51
majflt/s 0.00
VSZ 8057816
RSS 25328
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 7252
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52084
RSS 3480
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.3.7. Anomalies

No anomalies were detected during the course of this test.

3.2.3.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.4. Test ID: PVVP_BACK2BACK

3.2.4.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: pvvp

  • vswitchperf: GIT tag: d3607cb18d6d3f55b53c06479c748b0184ba0ac5

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • DPDK Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

  • VNF: QemuDpdkVhostUser, Version: 2.5.0, GIT tag: a8c40fa2d667e585382080db36ac44e216b37a1c

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: pvvp_back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: pvvp
  • Traffic type: rfc2544
  • Bidirectional : True
3.2.4.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.4.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1702
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:53:31
guest_loopback_app testpmd testpmd
3.2.4.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 35488
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 12.26
majflt/s 0.00
VSZ 16454616
RSS 25479
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 35472
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52076
RSS 3484
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 35858
%usr 0.00
%system 0.04
%guest 200.16
%CPU 100.15
CPU
minflt/s 1.36
majflt/s 0.00
VSZ 2648452
RSS 174801
%MEM 0.27
kB_rd/s 0.00
kB_wr/s 4.84
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 35731
%usr 0.00
%system 0.23
%guest 200.40
%CPU 100.17
CPU
minflt/s 1.30
majflt/s 0.00
VSZ 2656652
RSS 191715
%MEM 0.29
kB_rd/s 0.00
kB_wr/s 0.02
kB_ccwr/s 0.00
3.2.4.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.4.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 2416
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:53:31
guest_loopback_app testpmd testpmd
3.2.4.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 35488
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 12.26
majflt/s 0.00
VSZ 16454616
RSS 25479
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 35472
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52076
RSS 3484
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 35858
%usr 0.00
%system 0.04
%guest 200.16
%CPU 100.15
CPU
minflt/s 1.36
majflt/s 0.00
VSZ 2648452
RSS 174801
%MEM 0.27
kB_rd/s 0.00
kB_wr/s 4.84
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 35731
%usr 0.00
%system 0.23
%guest 200.40
%CPU 100.17
CPU
minflt/s 1.30
majflt/s 0.00
VSZ 2656652
RSS 191715
%MEM 0.29
kB_rd/s 0.00
kB_wr/s 0.02
kB_ccwr/s 0.00
3.2.4.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.4.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1075
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:53:31
guest_loopback_app testpmd testpmd
3.2.4.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 35488
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 12.26
majflt/s 0.00
VSZ 16454616
RSS 25479
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 35472
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52076
RSS 3484
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 35858
%usr 0.00
%system 0.04
%guest 200.16
%CPU 100.15
CPU
minflt/s 1.36
majflt/s 0.00
VSZ 2648452
RSS 174801
%MEM 0.27
kB_rd/s 0.00
kB_wr/s 4.84
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 35731
%usr 0.00
%system 0.23
%guest 200.40
%CPU 100.17
CPU
minflt/s 1.30
majflt/s 0.00
VSZ 2656652
RSS 191715
%MEM 0.29
kB_rd/s 0.00
kB_wr/s 0.02
kB_ccwr/s 0.00
3.2.4.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.4.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 35919454
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:53:31
guest_loopback_app testpmd testpmd
3.2.4.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 35488
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 12.26
majflt/s 0.00
VSZ 16454616
RSS 25479
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 35472
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52076
RSS 3484
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 35858
%usr 0.00
%system 0.04
%guest 200.16
%CPU 100.15
CPU
minflt/s 1.36
majflt/s 0.00
VSZ 2648452
RSS 174801
%MEM 0.27
kB_rd/s 0.00
kB_wr/s 4.84
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 35731
%usr 0.00
%system 0.23
%guest 200.40
%CPU 100.17
CPU
minflt/s 1.30
majflt/s 0.00
VSZ 2656652
RSS 191715
%MEM 0.29
kB_rd/s 0.00
kB_wr/s 0.02
kB_ccwr/s 0.00
3.2.4.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.4.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 24382314
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:53:31
guest_loopback_app testpmd testpmd
3.2.4.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 35488
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 12.26
majflt/s 0.00
VSZ 16454616
RSS 25479
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 35472
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52076
RSS 3484
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 35858
%usr 0.00
%system 0.04
%guest 200.16
%CPU 100.15
CPU
minflt/s 1.36
majflt/s 0.00
VSZ 2648452
RSS 174801
%MEM 0.27
kB_rd/s 0.00
kB_wr/s 4.84
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 35731
%usr 0.00
%system 0.23
%guest 200.40
%CPU 100.17
CPU
minflt/s 1.30
majflt/s 0.00
VSZ 2656652
RSS 191715
%MEM 0.29
kB_rd/s 0.00
kB_wr/s 0.02
kB_ccwr/s 0.00
3.2.4.7. Anomalies

No anomalies were detected during the course of this test.

3.2.4.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.5. Test ID: PHY2PHY_TPUT

3.2.5.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: d3607cb18d6d3f55b53c06479c748b0184ba0ac5

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • DPDK Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: phy2phy_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: p2p
  • Traffic type: rfc2544
  • Bidirectional : True
3.2.5.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.5.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 23086570.229
tx_rate_mbps Unknown
throughput_rx_mbps 11820.324
tx_rate_percent 77.570
throughput_rx_percent 77.570
frame_loss_percent 0.000
min_latency_ns 6980.000
max_latency_ns 355480.000
avg_latency_ns 8716.000
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:30:43
3.2.5.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 193057
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 23.59
majflt/s 0.00
VSZ 8057816
RSS 23241
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 193041
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52096
RSS 3460
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.5.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.5.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 15846779.936
tx_rate_mbps Unknown
throughput_rx_mbps 16227.103
tx_rate_percent 93.813
throughput_rx_percent 93.813
frame_loss_percent 0.000
min_latency_ns 5000.000
max_latency_ns 362700.000
avg_latency_ns 8969.500
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:30:43
3.2.5.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 193057
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 23.59
majflt/s 0.00
VSZ 8057816
RSS 23241
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 193041
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52096
RSS 3460
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.5.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.5.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699198.164
tx_rate_mbps Unknown
throughput_rx_mbps 19247.916
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 38900.000
max_latency_ns 71120.000
avg_latency_ns 60227.500
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:30:43
3.2.5.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 193057
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 23.59
majflt/s 0.00
VSZ 8057816
RSS 23241
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 193041
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52096
RSS 3460
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.5.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.5.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394611.882
tx_rate_mbps Unknown
throughput_rx_mbps 19616.661
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 40180.000
max_latency_ns 70960.000
avg_latency_ns 59838.500
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:30:43
3.2.5.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 193057
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 23.59
majflt/s 0.00
VSZ 8057816
RSS 23241
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 193041
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52096
RSS 3460
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.5.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.5.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625469.523
tx_rate_mbps Unknown
throughput_rx_mbps 19739.702
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 34660.000
max_latency_ns 68860.000
avg_latency_ns 57707.500
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:30:43
3.2.5.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 193057
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 23.59
majflt/s 0.00
VSZ 8057816
RSS 23241
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 193041
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52096
RSS 3460
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.5.7. Anomalies

No anomalies were detected during the course of this test.

3.2.5.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.6. Test ID: PHY2PHY_SCALABILITY

3.2.6.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: d3607cb18d6d3f55b53c06479c748b0184ba0ac5

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • DPDK Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: phy2phy_scalability
  • Description: LTD.Scalability.Flows.RFC2544.0PacketLoss
  • Deployment: p2p
  • Traffic type: rfc2544
  • Bidirectional : True
3.2.6.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.6.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 23316601.402
tx_rate_mbps Unknown
throughput_rx_mbps 11938.100
tx_rate_percent 78.344
throughput_rx_percent 78.344
frame_loss_percent 0.000
min_latency_ns 5700.000
max_latency_ns 88640.000
avg_latency_ns 9045.000
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:34:03
stream_count 8000
match_type L4
pre-installed_flows No
3.2.6.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 12621
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 12.12
majflt/s 0.00
VSZ 8057820
RSS 26961
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 12531
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52084
RSS 3480
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.6.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.6.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 15977430.495
tx_rate_mbps Unknown
throughput_rx_mbps 16360.889
tx_rate_percent 94.586
throughput_rx_percent 94.586
frame_loss_percent 0.000
min_latency_ns 5780.000
max_latency_ns 55060.000
avg_latency_ns 9062.000
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:34:03
stream_count 8000
match_type L4
pre-installed_flows No
3.2.6.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 12621
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 12.12
majflt/s 0.00
VSZ 8057820
RSS 26961
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 12531
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52084
RSS 3480
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.6.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.6.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699193.622
tx_rate_mbps Unknown
throughput_rx_mbps 19247.897
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 35020.000
max_latency_ns 70600.000
avg_latency_ns 59319.000
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:34:03
stream_count 8000
match_type L4
pre-installed_flows No
3.2.6.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 12621
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 12.12
majflt/s 0.00
VSZ 8057820
RSS 26961
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 12531
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52084
RSS 3480
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.6.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.6.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394608.761
tx_rate_mbps Unknown
throughput_rx_mbps 19616.635
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 33640.000
max_latency_ns 69060.000
avg_latency_ns 57290.000
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:34:03
stream_count 8000
match_type L4
pre-installed_flows No
3.2.6.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 12621
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 12.12
majflt/s 0.00
VSZ 8057820
RSS 26961
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 12531
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52084
RSS 3480
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.6.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.6.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625469.751
tx_rate_mbps Unknown
throughput_rx_mbps 19739.705
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 36680.000
max_latency_ns 70940.000
avg_latency_ns 59232.500
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:34:03
stream_count 8000
match_type L4
pre-installed_flows No
3.2.6.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 12621
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 12.12
majflt/s 0.00
VSZ 8057820
RSS 26961
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 12531
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52084
RSS 3480
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.6.7. Anomalies

No anomalies were detected during the course of this test.

3.2.6.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.7. Test ID: PVP_BACK2BACK

3.2.7.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: d3607cb18d6d3f55b53c06479c748b0184ba0ac5

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • DPDK Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

  • VNF: QemuDpdkVhostUser, Version: 2.5.0, GIT tag: a8c40fa2d667e585382080db36ac44e216b37a1c

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: pvp_back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: pvp
  • Traffic type: rfc2544
  • Bidirectional : True
3.2.7.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.7.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1277
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 01:01:12
guest_loopback_app testpmd
3.2.7.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 22240
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 15.74
majflt/s 0.00
VSZ 12256224
RSS 27403
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 22224
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52092
RSS 3460
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 22500
%usr 0.00
%system 0.13
%guest 200.36
%CPU 100.16
CPU
minflt/s 1.47
majflt/s 0.00
VSZ 2664848
RSS 260466
%MEM 0.40
kB_rd/s 0.00
kB_wr/s 3.94
kB_ccwr/s 0.00
3.2.7.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.7.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1449
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 01:01:12
guest_loopback_app testpmd
3.2.7.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 22240
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 15.74
majflt/s 0.00
VSZ 12256224
RSS 27403
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 22224
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52092
RSS 3460
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 22500
%usr 0.00
%system 0.13
%guest 200.36
%CPU 100.16
CPU
minflt/s 1.47
majflt/s 0.00
VSZ 2664848
RSS 260466
%MEM 0.40
kB_rd/s 0.00
kB_wr/s 3.94
kB_ccwr/s 0.00
3.2.7.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.7.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 40424262
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 01:01:12
guest_loopback_app testpmd
3.2.7.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 22240
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 15.74
majflt/s 0.00
VSZ 12256224
RSS 27403
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 22224
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52092
RSS 3460
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 22500
%usr 0.00
%system 0.13
%guest 200.36
%CPU 100.16
CPU
minflt/s 1.47
majflt/s 0.00
VSZ 2664848
RSS 260466
%MEM 0.40
kB_rd/s 0.00
kB_wr/s 3.94
kB_ccwr/s 0.00
3.2.7.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.7.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 35919540
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 01:01:12
guest_loopback_app testpmd
3.2.7.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 22240
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 15.74
majflt/s 0.00
VSZ 12256224
RSS 27403
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 22224
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52092
RSS 3460
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 22500
%usr 0.00
%system 0.13
%guest 200.36
%CPU 100.16
CPU
minflt/s 1.47
majflt/s 0.00
VSZ 2664848
RSS 260466
%MEM 0.40
kB_rd/s 0.00
kB_wr/s 3.94
kB_ccwr/s 0.00
3.2.7.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.7.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 24382311
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 01:01:12
guest_loopback_app testpmd
3.2.7.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 22240
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 15.74
majflt/s 0.00
VSZ 12256224
RSS 27403
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 22224
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52092
RSS 3460
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 22500
%usr 0.00
%system 0.13
%guest 200.36
%CPU 100.16
CPU
minflt/s 1.47
majflt/s 0.00
VSZ 2664848
RSS 260466
%MEM 0.40
kB_rd/s 0.00
kB_wr/s 3.94
kB_ccwr/s 0.00
3.2.7.7. Anomalies

No anomalies were detected during the course of this test.

3.2.7.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.8. Test ID: PVVP_TPUT

3.2.8.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: pvvp

  • vswitchperf: GIT tag: d3607cb18d6d3f55b53c06479c748b0184ba0ac5

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • DPDK Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

  • VNF: QemuDpdkVhostUser, Version: 2.5.0, GIT tag: a8c40fa2d667e585382080db36ac44e216b37a1c

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: pvvp_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvvp
  • Traffic type: rfc2544
  • Bidirectional : True
3.2.8.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.8.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 527808.869
tx_rate_mbps Unknown
throughput_rx_mbps 270.238
tx_rate_percent 1.773
throughput_rx_percent 1.773
frame_loss_percent 0.000
min_latency_ns 5200.000
max_latency_ns 34780.000
avg_latency_ns 5590.500
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:40:43
guest_loopback_app testpmd testpmd
3.2.8.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 30080
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 8.52
majflt/s 0.00
VSZ 16454616
RSS 25609
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 30064
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52080
RSS 3484
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 30348
%usr 0.00
%system 0.30
%guest 200.25
%CPU 100.17
CPU
minflt/s 1.63
majflt/s 0.00
VSZ 2604392
RSS 227068
%MEM 0.35
kB_rd/s 0.00
kB_wr/s 6.48
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 30323
%usr 0.00
%system 0.00
%guest 200.10
%CPU 100.16
CPU
minflt/s 0.11
majflt/s 0.00
VSZ 2651536
RSS 239932
%MEM 0.37
kB_rd/s 0.00
kB_wr/s 0.06
kB_ccwr/s 0.00
3.2.8.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.8.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 3827070.973
tx_rate_mbps Unknown
throughput_rx_mbps 3918.921
tx_rate_percent 22.656
throughput_rx_percent 22.656
frame_loss_percent 0.000
min_latency_ns 12780.000
max_latency_ns 134320.000
avg_latency_ns 19092.000
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:40:43
guest_loopback_app testpmd testpmd
3.2.8.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 30080
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 8.52
majflt/s 0.00
VSZ 16454616
RSS 25609
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 30064
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52080
RSS 3484
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 30348
%usr 0.00
%system 0.30
%guest 200.25
%CPU 100.17
CPU
minflt/s 1.63
majflt/s 0.00
VSZ 2604392
RSS 227068
%MEM 0.35
kB_rd/s 0.00
kB_wr/s 6.48
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 30323
%usr 0.00
%system 0.00
%guest 200.10
%CPU 100.16
CPU
minflt/s 0.11
majflt/s 0.00
VSZ 2651536
RSS 239932
%MEM 0.37
kB_rd/s 0.00
kB_wr/s 0.06
kB_ccwr/s 0.00
3.2.8.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.8.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 3536164.686
tx_rate_mbps Unknown
throughput_rx_mbps 14484.131
tx_rate_percent 75.250
throughput_rx_percent 75.250
frame_loss_percent 0.000
min_latency_ns 10480.000
max_latency_ns 113740.000
avg_latency_ns 34001.500
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:40:43
guest_loopback_app testpmd testpmd
3.2.8.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 30080
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 8.52
majflt/s 0.00
VSZ 16454616
RSS 25609
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 30064
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52080
RSS 3484
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 30348
%usr 0.00
%system 0.30
%guest 200.25
%CPU 100.17
CPU
minflt/s 1.63
majflt/s 0.00
VSZ 2604392
RSS 227068
%MEM 0.35
kB_rd/s 0.00
kB_wr/s 6.48
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 30323
%usr 0.00
%system 0.00
%guest 200.10
%CPU 100.16
CPU
minflt/s 0.11
majflt/s 0.00
VSZ 2651536
RSS 239932
%MEM 0.37
kB_rd/s 0.00
kB_wr/s 0.06
kB_ccwr/s 0.00
3.2.8.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.8.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2098294.927
tx_rate_mbps Unknown
throughput_rx_mbps 17189.232
tx_rate_percent 87.625
throughput_rx_percent 87.625
frame_loss_percent 0.000
min_latency_ns 11580.000
max_latency_ns 89000.000
avg_latency_ns 22182.000
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:40:43
guest_loopback_app testpmd testpmd
3.2.8.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 30080
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 8.52
majflt/s 0.00
VSZ 16454616
RSS 25609
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 30064
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52080
RSS 3484
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 30348
%usr 0.00
%system 0.30
%guest 200.25
%CPU 100.17
CPU
minflt/s 1.63
majflt/s 0.00
VSZ 2604392
RSS 227068
%MEM 0.35
kB_rd/s 0.00
kB_wr/s 6.48
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 30323
%usr 0.00
%system 0.00
%guest 200.10
%CPU 100.16
CPU
minflt/s 0.11
majflt/s 0.00
VSZ 2651536
RSS 239932
%MEM 0.37
kB_rd/s 0.00
kB_wr/s 0.06
kB_ccwr/s 0.00
3.2.8.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.8.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1449478.478
tx_rate_mbps Unknown
throughput_rx_mbps 17602.467
tx_rate_percent 89.172
throughput_rx_percent 89.172
frame_loss_percent 0.000
min_latency_ns 14700.000
max_latency_ns 174040.000
avg_latency_ns 18565.000
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:40:43
guest_loopback_app testpmd testpmd
3.2.8.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 30080
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 8.52
majflt/s 0.00
VSZ 16454616
RSS 25609
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 30064
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52080
RSS 3484
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 30348
%usr 0.00
%system 0.30
%guest 200.25
%CPU 100.17
CPU
minflt/s 1.63
majflt/s 0.00
VSZ 2604392
RSS 227068
%MEM 0.35
kB_rd/s 0.00
kB_wr/s 6.48
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 30323
%usr 0.00
%system 0.00
%guest 200.10
%CPU 100.16
CPU
minflt/s 0.11
majflt/s 0.00
VSZ 2651536
RSS 239932
%MEM 0.37
kB_rd/s 0.00
kB_wr/s 0.06
kB_ccwr/s 0.00
3.2.8.7. Anomalies

No anomalies were detected during the course of this test.

3.2.8.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.3. Rationale for decisions

The tests conducted do not have pass/fail/conditional-pass criteria. The test is simply conducted and the results are reported.

3.4. Conclusions and recommendations

The test results are stable. The vsperf CI jobs that were used to obtain the results can be found at https://artifacts.opnfv.org/logs/vswitchperf/intel-pod3/2016-12-06_21-06-09/vswitchperf_logs_2016-12-06_21-06-09.tar.gz.

3.5. General

3.5.1. Glossary

  • NFV - Network Function Virtualization
  • Mbps - 1,000,000bps

3.5.2. Document change procedures and history

Document ID Author Date Modified
OPNFV_vswitchperf_LTR_ver_1.0_Jan_15_CN_DRAFT Christopher Nolan 23/01/2015
OPNFV_vswitchperf_LTR_ver_1.1_Jan_15_CN_DRAFT Christopher Nolan 28/01/2015

4. Performance report for Open vSwitch

4.1. Introduction

The objective of the OPNFV project titled “Characterise vSwitch Performance for Telco NFV Use Cases”, is to evaluate a virtual switch to identify its suitability for a Telco Network Function Virtualization (NFV) environment. As well as this, the project aims to identify any gaps or bottlenecks in order to drive architectural changes to improve virtual switch performance and determinism. The purpose of this document is to summarize the results of the tests carried out on the virtual switch in the Network Function Virtualization Infrastructure (NFVI) and, from these results, provide evaluations and recommendations for the virtual switch. Test results will be outlined in details-of-LTR, preceded by the document-identifier and the scope and references).

This document is currently in draft form.

4.1.1. Document identifier

The document id will be used to uniquely identify versions of the LTR. The format for the document id will be: OPNFV_vswitchperf_LTR_rel_STATUS, the status is one of: DRAFT, REVIEWED, CORRECTED or FINAL. The document id for this version of the LTR is: OPNFV_vswitchperf_LTR_Brahmaputra_DRAFT.

4.1.2. Scope

The scope of this report is to detail the results of the tests that have been performed on the virtual switch. This report will also evaluate the results of these tests and, based on these evaluations, provide recommendations on the suitability of the virtual switch for use in a Telco NFV environment.

4.2. Details of the Level Test Report

This section provides a test-results-overview. Also included are the rationale and the conclusions.

4.2.1. Test ID: BACK2BACK

4.2.1.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: d3607cb18d6d3f55b53c06479c748b0184ba0ac5

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

Below are test details:

  • Test ID: back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: p2p
  • Traffic type: rfc2544
  • Bidirectional : True
4.2.1.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.1.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 5108
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:46:18
4.2.1.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 47627
%usr 0.35
%system 4.47
%guest 0.00
%CPU 4.82
CPU
minflt/s 5.43
majflt/s 0.00
VSZ 6609928
RSS 31446
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 47624
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45904
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.1.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 3382
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:46:18
4.2.1.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 47627
%usr 0.35
%system 4.47
%guest 0.00
%CPU 4.82
CPU
minflt/s 5.43
majflt/s 0.00
VSZ 6609928
RSS 31446
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 47624
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45904
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.1.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1344
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:46:18
4.2.1.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 47627
%usr 0.35
%system 4.47
%guest 0.00
%CPU 4.82
CPU
minflt/s 5.43
majflt/s 0.00
VSZ 6609928
RSS 31446
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 47624
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45904
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.1.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1096
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:46:18
4.2.1.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 47627
%usr 0.35
%system 4.47
%guest 0.00
%CPU 4.82
CPU
minflt/s 5.43
majflt/s 0.00
VSZ 6609928
RSS 31446
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 47624
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45904
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.1.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 744
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:46:18
4.2.1.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 47627
%usr 0.35
%system 4.47
%guest 0.00
%CPU 4.82
CPU
minflt/s 5.43
majflt/s 0.00
VSZ 6609928
RSS 31446
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 47624
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45904
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.7. Anomalies

No anomalies were detected during the course of this test.

4.2.1.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.2. Test ID: PVP_TPUT

4.2.2.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: d3607cb18d6d3f55b53c06479c748b0184ba0ac5

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • VNF: QemuVirtioNet, Version: 2.5.0, GIT tag: a8c40fa2d667e585382080db36ac44e216b37a1c

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: pvp_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvp
  • Traffic type: rfc2544
  • Bidirectional : True
4.2.2.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.2.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 501227.444
tx_rate_mbps Unknown
throughput_rx_mbps 256.628
tx_rate_percent 1.773
throughput_rx_percent 1.684
frame_loss_percent 5.131
min_latency_ns 11520.000
max_latency_ns 56946100.000
avg_latency_ns 229416.500
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:42:52
guest_loopback_app testpmd
4.2.2.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 63479
%usr 0.33
%system 4.30
%guest 0.00
%CPU 4.63
CPU
minflt/s 5.50
majflt/s 0.00
VSZ 6609928
RSS 33997
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 63476
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 63764
%usr 0.00
%system 0.68
%guest 199.89
%CPU 100.17
CPU
minflt/s 1.73
majflt/s 0.00
VSZ 2695692
RSS 300987
%MEM 0.46
kB_rd/s 0.00
kB_wr/s 5.78
kB_ccwr/s 0.00
4.2.2.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.2.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 299469.619
tx_rate_mbps Unknown
throughput_rx_mbps 306.657
tx_rate_percent 1.773
throughput_rx_percent 1.773
frame_loss_percent 0.033
min_latency_ns 11760.000
max_latency_ns 14474540.000
avg_latency_ns 48711.500
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:42:52
guest_loopback_app testpmd
4.2.2.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 63479
%usr 0.33
%system 4.30
%guest 0.00
%CPU 4.63
CPU
minflt/s 5.50
majflt/s 0.00
VSZ 6609928
RSS 33997
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 63476
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 63764
%usr 0.00
%system 0.68
%guest 199.89
%CPU 100.17
CPU
minflt/s 1.73
majflt/s 0.00
VSZ 2695692
RSS 300987
%MEM 0.46
kB_rd/s 0.00
kB_wr/s 5.78
kB_ccwr/s 0.00
4.2.2.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.2.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 156030.018
tx_rate_mbps Unknown
throughput_rx_mbps 639.099
tx_rate_percent 3.320
throughput_rx_percent 3.320
frame_loss_percent 0.000
min_latency_ns 15940.000
max_latency_ns 1905180.000
avg_latency_ns 93368.000
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:42:52
guest_loopback_app testpmd
4.2.2.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 63479
%usr 0.33
%system 4.30
%guest 0.00
%CPU 4.63
CPU
minflt/s 5.50
majflt/s 0.00
VSZ 6609928
RSS 33997
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 63476
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 63764
%usr 0.00
%system 0.68
%guest 199.89
%CPU 100.17
CPU
minflt/s 1.73
majflt/s 0.00
VSZ 2695692
RSS 300987
%MEM 0.46
kB_rd/s 0.00
kB_wr/s 5.78
kB_ccwr/s 0.00
4.2.2.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.2.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 135072.570
tx_rate_mbps Unknown
throughput_rx_mbps 1106.514
tx_rate_percent 5.641
throughput_rx_percent 5.641
frame_loss_percent 0.000
min_latency_ns 21740.000
max_latency_ns 1382260.000
avg_latency_ns 91060.500
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:42:52
guest_loopback_app testpmd
4.2.2.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 63479
%usr 0.33
%system 4.30
%guest 0.00
%CPU 4.63
CPU
minflt/s 5.50
majflt/s 0.00
VSZ 6609928
RSS 33997
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 63476
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 63764
%usr 0.00
%system 0.68
%guest 199.89
%CPU 100.17
CPU
minflt/s 1.73
majflt/s 0.00
VSZ 2695692
RSS 300987
%MEM 0.46
kB_rd/s 0.00
kB_wr/s 5.78
kB_ccwr/s 0.00
4.2.2.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.2.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 217415.162
tx_rate_mbps Unknown
throughput_rx_mbps 2640.290
tx_rate_percent 13.375
throughput_rx_percent 13.375
frame_loss_percent 0.000
min_latency_ns 33640.000
max_latency_ns 2059200.000
avg_latency_ns 114331.000
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:42:52
guest_loopback_app testpmd
4.2.2.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 63479
%usr 0.33
%system 4.30
%guest 0.00
%CPU 4.63
CPU
minflt/s 5.50
majflt/s 0.00
VSZ 6609928
RSS 33997
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 63476
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 63764
%usr 0.00
%system 0.68
%guest 199.89
%CPU 100.17
CPU
minflt/s 1.73
majflt/s 0.00
VSZ 2695692
RSS 300987
%MEM 0.46
kB_rd/s 0.00
kB_wr/s 5.78
kB_ccwr/s 0.00
4.2.2.7. Anomalies

No anomalies were detected during the course of this test.

4.2.2.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.3. Test ID: PHY2PHY_TPUT_MOD_VLAN

4.2.3.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: d3607cb18d6d3f55b53c06479c748b0184ba0ac5

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

Below are test details:

  • Test ID: phy2phy_tput_mod_vlan
  • Description: LTD.Throughput.RFC2544.PacketLossRatioFrameModification
  • Deployment: p2p
  • Traffic type: rfc2544
  • Bidirectional : False
4.2.3.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.3.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 724284.427
tx_rate_mbps Unknown
throughput_rx_mbps 394.011
tx_rate_percent 4.867
throughput_rx_percent 5.099
frame_loss_percent 0.000
min_latency_ns 4160.000
max_latency_ns 5439240.000
avg_latency_ns 380156.000
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:39:39
4.2.3.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 53366
%usr 0.35
%system 4.52
%guest 0.00
%CPU 4.87
CPU
minflt/s 5.55
majflt/s 0.00
VSZ 6609932
RSS 33091
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 53335
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3255
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.3.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.3.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 672377.707
tx_rate_mbps Unknown
throughput_rx_mbps 710.031
tx_rate_percent 7.961
throughput_rx_percent 8.176
frame_loss_percent 0.000
min_latency_ns 4420.000
max_latency_ns 3501580.000
avg_latency_ns 200408.000
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:39:39
4.2.3.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 53366
%usr 0.35
%system 4.52
%guest 0.00
%CPU 4.87
CPU
minflt/s 5.55
majflt/s 0.00
VSZ 6609932
RSS 33091
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 53335
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3255
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.3.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.3.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 568682.538
tx_rate_mbps Unknown
throughput_rx_mbps 2347.522
tx_rate_percent 24.203
throughput_rx_percent 24.385
frame_loss_percent 0.000
min_latency_ns 5380.000
max_latency_ns 1560420.000
avg_latency_ns 85225.000
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:39:39
4.2.3.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 53366
%usr 0.35
%system 4.52
%guest 0.00
%CPU 4.87
CPU
minflt/s 5.55
majflt/s 0.00
VSZ 6609932
RSS 33091
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 53335
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3255
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.3.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.3.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 567603.421
tx_rate_mbps Unknown
throughput_rx_mbps 4667.971
tx_rate_percent 47.406
throughput_rx_percent 47.588
frame_loss_percent 0.000
min_latency_ns 5840.000
max_latency_ns 1191300.000
avg_latency_ns 78658.000
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:39:39
4.2.3.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 53366
%usr 0.35
%system 4.52
%guest 0.00
%CPU 4.87
CPU
minflt/s 5.55
majflt/s 0.00
VSZ 6609932
RSS 33091
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 53335
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3255
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.3.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.3.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 567587.389
tx_rate_mbps Unknown
throughput_rx_mbps 6910.944
tx_rate_percent 69.836
throughput_rx_percent 70.018
frame_loss_percent 0.000
min_latency_ns 6380.000
max_latency_ns 1291200.000
avg_latency_ns 79138.000
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:39:39
4.2.3.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 53366
%usr 0.35
%system 4.52
%guest 0.00
%CPU 4.87
CPU
minflt/s 5.55
majflt/s 0.00
VSZ 6609932
RSS 33091
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 53335
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3255
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.3.7. Anomalies

No anomalies were detected during the course of this test.

4.2.3.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.4. Test ID: PVVP_BACK2BACK

4.2.4.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: pvvp

  • vswitchperf: GIT tag: d3607cb18d6d3f55b53c06479c748b0184ba0ac5

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • VNF: QemuVirtioNet, Version: 2.5.0, GIT tag: a8c40fa2d667e585382080db36ac44e216b37a1c

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: pvvp_back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: pvvp
  • Traffic type: rfc2544
  • Bidirectional : True
4.2.4.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.4.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:48:49
guest_loopback_app testpmd testpmd
4.2.4.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 80443
%usr 0.34
%system 4.30
%guest 0.00
%CPU 4.64
CPU
minflt/s 7.69
majflt/s 0.00
VSZ 6610052
RSS 34128
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 80440
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 80932
%usr 0.00
%system 0.25
%guest 200.41
%CPU 100.17
CPU
minflt/s 1.39
majflt/s 0.00
VSZ 2692632
RSS 243071
%MEM 0.37
kB_rd/s 0.00
kB_wr/s 5.05
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 80806
%usr 0.00
%system 0.05
%guest 200.09
%CPU 100.16
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2611740
RSS 228135
%MEM 0.35
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
4.2.4.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.4.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:48:49
guest_loopback_app testpmd testpmd
4.2.4.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 80443
%usr 0.34
%system 4.30
%guest 0.00
%CPU 4.64
CPU
minflt/s 7.69
majflt/s 0.00
VSZ 6610052
RSS 34128
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 80440
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 80932
%usr 0.00
%system 0.25
%guest 200.41
%CPU 100.17
CPU
minflt/s 1.39
majflt/s 0.00
VSZ 2692632
RSS 243071
%MEM 0.37
kB_rd/s 0.00
kB_wr/s 5.05
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 80806
%usr 0.00
%system 0.05
%guest 200.09
%CPU 100.16
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2611740
RSS 228135
%MEM 0.35
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
4.2.4.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.4.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:48:49
guest_loopback_app testpmd testpmd
4.2.4.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 80443
%usr 0.34
%system 4.30
%guest 0.00
%CPU 4.64
CPU
minflt/s 7.69
majflt/s 0.00
VSZ 6610052
RSS 34128
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 80440
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 80932
%usr 0.00
%system 0.25
%guest 200.41
%CPU 100.17
CPU
minflt/s 1.39
majflt/s 0.00
VSZ 2692632
RSS 243071
%MEM 0.37
kB_rd/s 0.00
kB_wr/s 5.05
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 80806
%usr 0.00
%system 0.05
%guest 200.09
%CPU 100.16
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2611740
RSS 228135
%MEM 0.35
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
4.2.4.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.4.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:48:49
guest_loopback_app testpmd testpmd
4.2.4.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 80443
%usr 0.34
%system 4.30
%guest 0.00
%CPU 4.64
CPU
minflt/s 7.69
majflt/s 0.00
VSZ 6610052
RSS 34128
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 80440
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 80932
%usr 0.00
%system 0.25
%guest 200.41
%CPU 100.17
CPU
minflt/s 1.39
majflt/s 0.00
VSZ 2692632
RSS 243071
%MEM 0.37
kB_rd/s 0.00
kB_wr/s 5.05
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 80806
%usr 0.00
%system 0.05
%guest 200.09
%CPU 100.16
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2611740
RSS 228135
%MEM 0.35
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
4.2.4.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.4.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:48:49
guest_loopback_app testpmd testpmd
4.2.4.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 80443
%usr 0.34
%system 4.30
%guest 0.00
%CPU 4.64
CPU
minflt/s 7.69
majflt/s 0.00
VSZ 6610052
RSS 34128
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 80440
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 80932
%usr 0.00
%system 0.25
%guest 200.41
%CPU 100.17
CPU
minflt/s 1.39
majflt/s 0.00
VSZ 2692632
RSS 243071
%MEM 0.37
kB_rd/s 0.00
kB_wr/s 5.05
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 80806
%usr 0.00
%system 0.05
%guest 200.09
%CPU 100.16
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2611740
RSS 228135
%MEM 0.35
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
4.2.4.7. Anomalies

No anomalies were detected during the course of this test.

4.2.4.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.5. Test ID: PHY2PHY_TPUT

4.2.5.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: d3607cb18d6d3f55b53c06479c748b0184ba0ac5

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

Below are test details:

  • Test ID: phy2phy_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: p2p
  • Traffic type: rfc2544
  • Bidirectional : True
4.2.5.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.5.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 758000.763
tx_rate_mbps Unknown
throughput_rx_mbps 388.096
tx_rate_percent 2.547
throughput_rx_percent 2.547
frame_loss_percent 0.000
min_latency_ns 4020.000
max_latency_ns 6905820.000
avg_latency_ns 476357.000
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:38:27
4.2.5.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 42792
%usr 0.36
%system 4.37
%guest 0.00
%CPU 4.72
CPU
minflt/s 5.63
majflt/s 0.00
VSZ 6609932
RSS 31453
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 42789
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.5.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.5.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 691513.141
tx_rate_mbps Unknown
throughput_rx_mbps 708.109
tx_rate_percent 4.094
throughput_rx_percent 4.094
frame_loss_percent 0.000
min_latency_ns 4260.000
max_latency_ns 3676660.000
avg_latency_ns 195448.500
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:38:27
4.2.5.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 42792
%usr 0.36
%system 4.37
%guest 0.00
%CPU 4.72
CPU
minflt/s 5.63
majflt/s 0.00
VSZ 6609932
RSS 31453
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 42789
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.5.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.5.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 628534.576
tx_rate_mbps Unknown
throughput_rx_mbps 2574.478
tx_rate_percent 13.375
throughput_rx_percent 13.375
frame_loss_percent 0.000
min_latency_ns 5280.000
max_latency_ns 2866900.000
avg_latency_ns 156704.500
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:38:27
4.2.5.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 42792
%usr 0.36
%system 4.37
%guest 0.00
%CPU 4.72
CPU
minflt/s 5.63
majflt/s 0.00
VSZ 6609932
RSS 31453
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 42789
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.5.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.5.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 579577.758
tx_rate_mbps Unknown
throughput_rx_mbps 4747.901
tx_rate_percent 24.203
throughput_rx_percent 24.203
frame_loss_percent 0.000
min_latency_ns 5820.000
max_latency_ns 1896620.000
avg_latency_ns 88725.000
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:38:27
4.2.5.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 42792
%usr 0.36
%system 4.37
%guest 0.00
%CPU 4.72
CPU
minflt/s 5.63
majflt/s 0.00
VSZ 6609932
RSS 31453
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 42789
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.5.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.5.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 506569.183
tx_rate_mbps Unknown
throughput_rx_mbps 6151.776
tx_rate_percent 31.164
throughput_rx_percent 31.164
frame_loss_percent 0.000
min_latency_ns 6600.000
max_latency_ns 560200.000
avg_latency_ns 45720.500
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:38:27
4.2.5.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 42792
%usr 0.36
%system 4.37
%guest 0.00
%CPU 4.72
CPU
minflt/s 5.63
majflt/s 0.00
VSZ 6609932
RSS 31453
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 42789
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.5.7. Anomalies

No anomalies were detected during the course of this test.

4.2.5.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.6. Test ID: PHY2PHY_SCALABILITY

4.2.6.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: d3607cb18d6d3f55b53c06479c748b0184ba0ac5

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

Below are test details:

  • Test ID: phy2phy_scalability
  • Description: LTD.Scalability.Flows.RFC2544.0PacketLoss
  • Deployment: p2p
  • Traffic type: rfc2544
  • Bidirectional : True
4.2.6.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.6.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 758000.313
tx_rate_mbps Unknown
throughput_rx_mbps 388.096
tx_rate_percent 2.547
throughput_rx_percent 2.547
frame_loss_percent 0.000
min_latency_ns 4040.000
max_latency_ns 7908560.000
avg_latency_ns 913425.500
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:41:11
stream_count 8000
match_type L4
pre-installed_flows No
4.2.6.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 58368
%usr 0.35
%system 4.39
%guest 0.00
%CPU 4.74
CPU
minflt/s 5.85
majflt/s 0.00
VSZ 6609928
RSS 34719
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 58365
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.6.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.6.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 691512.843
tx_rate_mbps Unknown
throughput_rx_mbps 708.109
tx_rate_percent 4.094
throughput_rx_percent 4.094
frame_loss_percent 0.000
min_latency_ns 4320.000
max_latency_ns 4134920.000
avg_latency_ns 260786.000
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:41:11
stream_count 8000
match_type L4
pre-installed_flows No
4.2.6.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 58368
%usr 0.35
%system 4.39
%guest 0.00
%CPU 4.74
CPU
minflt/s 5.85
majflt/s 0.00
VSZ 6609928
RSS 34719
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 58365
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.6.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.6.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 628548.407
tx_rate_mbps Unknown
throughput_rx_mbps 2574.534
tx_rate_percent 13.375
throughput_rx_percent 13.375
frame_loss_percent 0.000
min_latency_ns 5260.000
max_latency_ns 4024620.000
avg_latency_ns 284135.500
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:41:11
stream_count 8000
match_type L4
pre-installed_flows No
4.2.6.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 58368
%usr 0.35
%system 4.39
%guest 0.00
%CPU 4.74
CPU
minflt/s 5.85
majflt/s 0.00
VSZ 6609928
RSS 34719
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 58365
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.6.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.6.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 598097.900
tx_rate_mbps Unknown
throughput_rx_mbps 4899.618
tx_rate_percent 24.977
throughput_rx_percent 24.977
frame_loss_percent 0.000
min_latency_ns 5620.000
max_latency_ns 2557880.000
avg_latency_ns 163671.000
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:41:11
stream_count 8000
match_type L4
pre-installed_flows No
4.2.6.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 58368
%usr 0.35
%system 4.39
%guest 0.00
%CPU 4.74
CPU
minflt/s 5.85
majflt/s 0.00
VSZ 6609928
RSS 34719
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 58365
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.6.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.6.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 582001.304
tx_rate_mbps Unknown
throughput_rx_mbps 7067.824
tx_rate_percent 35.805
throughput_rx_percent 35.805
frame_loss_percent 0.000
min_latency_ns 6500.000
max_latency_ns 2682860.000
avg_latency_ns 134698.500
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:41:11
stream_count 8000
match_type L4
pre-installed_flows No
4.2.6.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 58368
%usr 0.35
%system 4.39
%guest 0.00
%CPU 4.74
CPU
minflt/s 5.85
majflt/s 0.00
VSZ 6609928
RSS 34719
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 58365
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.6.7. Anomalies

No anomalies were detected during the course of this test.

4.2.6.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.7. Test ID: PVP_BACK2BACK

4.2.7.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: d3607cb18d6d3f55b53c06479c748b0184ba0ac5

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • VNF: QemuVirtioNet, Version: 2.5.0, GIT tag: a8c40fa2d667e585382080db36ac44e216b37a1c

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: pvp_back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: pvp
  • Traffic type: rfc2544
  • Bidirectional : True
4.2.7.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.7.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:46:44
guest_loopback_app testpmd
4.2.7.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 69011
%usr 0.34
%system 4.36
%guest 0.00
%CPU 4.70
CPU
minflt/s 6.34
majflt/s 0.00
VSZ 6609928
RSS 32194
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 69008
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3260
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 69300
%usr 0.00
%system 0.19
%guest 200.29
%CPU 100.16
CPU
minflt/s 1.47
majflt/s 0.00
VSZ 2677276
RSS 199461
%MEM 0.30
kB_rd/s 0.00
kB_wr/s 5.33
kB_ccwr/s 0.00
4.2.7.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.7.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:46:44
guest_loopback_app testpmd
4.2.7.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 69011
%usr 0.34
%system 4.36
%guest 0.00
%CPU 4.70
CPU
minflt/s 6.34
majflt/s 0.00
VSZ 6609928
RSS 32194
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 69008
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3260
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 69300
%usr 0.00
%system 0.19
%guest 200.29
%CPU 100.16
CPU
minflt/s 1.47
majflt/s 0.00
VSZ 2677276
RSS 199461
%MEM 0.30
kB_rd/s 0.00
kB_wr/s 5.33
kB_ccwr/s 0.00
4.2.7.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.7.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:46:44
guest_loopback_app testpmd
4.2.7.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 69011
%usr 0.34
%system 4.36
%guest 0.00
%CPU 4.70
CPU
minflt/s 6.34
majflt/s 0.00
VSZ 6609928
RSS 32194
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 69008
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3260
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 69300
%usr 0.00
%system 0.19
%guest 200.29
%CPU 100.16
CPU
minflt/s 1.47
majflt/s 0.00
VSZ 2677276
RSS 199461
%MEM 0.30
kB_rd/s 0.00
kB_wr/s 5.33
kB_ccwr/s 0.00
4.2.7.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.7.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:46:44
guest_loopback_app testpmd
4.2.7.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 69011
%usr 0.34
%system 4.36
%guest 0.00
%CPU 4.70
CPU
minflt/s 6.34
majflt/s 0.00
VSZ 6609928
RSS 32194
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 69008
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3260
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 69300
%usr 0.00
%system 0.19
%guest 200.29
%CPU 100.16
CPU
minflt/s 1.47
majflt/s 0.00
VSZ 2677276
RSS 199461
%MEM 0.30
kB_rd/s 0.00
kB_wr/s 5.33
kB_ccwr/s 0.00
4.2.7.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.7.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:46:44
guest_loopback_app testpmd
4.2.7.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 69011
%usr 0.34
%system 4.36
%guest 0.00
%CPU 4.70
CPU
minflt/s 6.34
majflt/s 0.00
VSZ 6609928
RSS 32194
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 69008
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3260
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 69300
%usr 0.00
%system 0.19
%guest 200.29
%CPU 100.16
CPU
minflt/s 1.47
majflt/s 0.00
VSZ 2677276
RSS 199461
%MEM 0.30
kB_rd/s 0.00
kB_wr/s 5.33
kB_ccwr/s 0.00
4.2.7.7. Anomalies

No anomalies were detected during the course of this test.

4.2.7.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.8. Test ID: PVVP_TPUT

4.2.8.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: pvvp

  • vswitchperf: GIT tag: d3607cb18d6d3f55b53c06479c748b0184ba0ac5

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • VNF: QemuVirtioNet, Version: 2.5.0, GIT tag: a8c40fa2d667e585382080db36ac44e216b37a1c

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: pvvp_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvvp
  • Traffic type: rfc2544
  • Bidirectional : True
4.2.8.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.8.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 523451.585
tx_rate_mbps Unknown
throughput_rx_mbps 268.007
tx_rate_percent 1.773
throughput_rx_percent 1.759
frame_loss_percent 0.826
min_latency_ns 22620.000
max_latency_ns 30649380.000
avg_latency_ns 139156.500
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:41:39
guest_loopback_app testpmd testpmd
4.2.8.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 74971
%usr 0.35
%system 4.30
%guest 0.00
%CPU 4.65
CPU
minflt/s 8.57
majflt/s 0.00
VSZ 6610056
RSS 32462
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 74968
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3016
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 75350
%usr 0.00
%system 0.62
%guest 199.68
%CPU 100.15
CPU
minflt/s 1.48
majflt/s 0.00
VSZ 2590236
RSS 127301
%MEM 0.19
kB_rd/s 0.00
kB_wr/s 5.91
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 75326
%usr 0.00
%system 0.51
%guest 199.60
%CPU 100.16
CPU
minflt/s 0.13
majflt/s 0.00
VSZ 2663956
RSS 168192
%MEM 0.26
kB_rd/s 0.00
kB_wr/s 0.06
kB_ccwr/s 0.00
4.2.8.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.8.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 299549.455
tx_rate_mbps Unknown
throughput_rx_mbps 306.739
tx_rate_percent 1.773
throughput_rx_percent 1.773
frame_loss_percent 0.006
min_latency_ns 19360.000
max_latency_ns 2528360.000
avg_latency_ns 60695.500
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:41:39
guest_loopback_app testpmd testpmd
4.2.8.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 74971
%usr 0.35
%system 4.30
%guest 0.00
%CPU 4.65
CPU
minflt/s 8.57
majflt/s 0.00
VSZ 6610056
RSS 32462
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 74968
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3016
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 75350
%usr 0.00
%system 0.62
%guest 199.68
%CPU 100.15
CPU
minflt/s 1.48
majflt/s 0.00
VSZ 2590236
RSS 127301
%MEM 0.19
kB_rd/s 0.00
kB_wr/s 5.91
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 75326
%usr 0.00
%system 0.51
%guest 199.60
%CPU 100.16
CPU
minflt/s 0.13
majflt/s 0.00
VSZ 2663956
RSS 168192
%MEM 0.26
kB_rd/s 0.00
kB_wr/s 0.06
kB_ccwr/s 0.00
4.2.8.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.8.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 228721.732
tx_rate_mbps Unknown
throughput_rx_mbps 936.844
tx_rate_percent 4.867
throughput_rx_percent 4.867
frame_loss_percent 0.000
min_latency_ns 44100.000
max_latency_ns 384140.000
avg_latency_ns 121254.500
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:41:39
guest_loopback_app testpmd testpmd
4.2.8.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 74971
%usr 0.35
%system 4.30
%guest 0.00
%CPU 4.65
CPU
minflt/s 8.57
majflt/s 0.00
VSZ 6610056
RSS 32462
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 74968
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3016
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 75350
%usr 0.00
%system 0.62
%guest 199.68
%CPU 100.15
CPU
minflt/s 1.48
majflt/s 0.00
VSZ 2590236
RSS 127301
%MEM 0.19
kB_rd/s 0.00
kB_wr/s 5.91
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 75326
%usr 0.00
%system 0.51
%guest 199.60
%CPU 100.16
CPU
minflt/s 0.13
majflt/s 0.00
VSZ 2663956
RSS 168192
%MEM 0.26
kB_rd/s 0.00
kB_wr/s 0.06
kB_ccwr/s 0.00
4.2.8.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.8.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 190636.072
tx_rate_mbps Unknown
throughput_rx_mbps 1561.691
tx_rate_percent 7.961
throughput_rx_percent 7.961
frame_loss_percent 0.000
min_latency_ns 54080.000
max_latency_ns 304940.000
avg_latency_ns 122739.000
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:41:39
guest_loopback_app testpmd testpmd
4.2.8.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 74971
%usr 0.35
%system 4.30
%guest 0.00
%CPU 4.65
CPU
minflt/s 8.57
majflt/s 0.00
VSZ 6610056
RSS 32462
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 74968
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3016
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 75350
%usr 0.00
%system 0.62
%guest 199.68
%CPU 100.15
CPU
minflt/s 1.48
majflt/s 0.00
VSZ 2590236
RSS 127301
%MEM 0.19
kB_rd/s 0.00
kB_wr/s 5.91
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 75326
%usr 0.00
%system 0.51
%guest 199.60
%CPU 100.16
CPU
minflt/s 0.13
majflt/s 0.00
VSZ 2663956
RSS 168192
%MEM 0.26
kB_rd/s 0.00
kB_wr/s 0.06
kB_ccwr/s 0.00
4.2.8.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.8.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 267698.795
tx_rate_mbps Unknown
throughput_rx_mbps 3250.934
tx_rate_percent 16.469
throughput_rx_percent 16.469
frame_loss_percent 0.000
min_latency_ns 0.000
max_latency_ns 1113940.000
avg_latency_ns 140594.000
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:41:39
guest_loopback_app testpmd testpmd
4.2.8.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 74971
%usr 0.35
%system 4.30
%guest 0.00
%CPU 4.65
CPU
minflt/s 8.57
majflt/s 0.00
VSZ 6610056
RSS 32462
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 74968
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3016
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 75350
%usr 0.00
%system 0.62
%guest 199.68
%CPU 100.15
CPU
minflt/s 1.48
majflt/s 0.00
VSZ 2590236
RSS 127301
%MEM 0.19
kB_rd/s 0.00
kB_wr/s 5.91
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 75326
%usr 0.00
%system 0.51
%guest 199.60
%CPU 100.16
CPU
minflt/s 0.13
majflt/s 0.00
VSZ 2663956
RSS 168192
%MEM 0.26
kB_rd/s 0.00
kB_wr/s 0.06
kB_ccwr/s 0.00
4.2.8.7. Anomalies

No anomalies were detected during the course of this test.

4.2.8.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.3. Rationale for decisions

The tests conducted do not have pass/fail/conditional-pass criteria. The test is simply conducted and the results are reported.

4.4. Conclusions and recommendations

The test results are stable. The vsperf CI jobs that were used to obtain the results can be found at https://artifacts.opnfv.org/logs/vswitchperf/intel-pod3/2016-12-06_21-06-09/vswitchperf_logs_2016-12-06_21-06-09.tar.gz.

4.5. General

4.5.1. Glossary

  • NFV - Network Function Virtualization
  • Mbps - 1,000,000bps

4.5.2. Document change procedures and history

Document ID Author Date Modified
OPNFV_vswitchperf_LTR_ver_1.0_Jan_15_CN_DRAFT Christopher Nolan 23/01/2015
OPNFV_vswitchperf_LTR_ver_1.1_Jan_15_CN_DRAFT Christopher Nolan 28/01/2015