VSPERF Results

1. OPNFV Brahmaputra Scenarios

Available Tests and aspects of scenarios:

Framework Test Definition
phy2phy_tput PacketLossRatio for Phy2Phy
back2back BackToBackFrames for Phy2Phy
phy2phy_tput_mod_vlan PacketLossRatioFrameModification for Phy2Phy
phy2phy_cont Phy2Phy blast vswitch at x% TX rate and measure throughput
pvp_cont PVP blast vswitch at x% TX rate and measure throughput
pvvp_cont PVVP blast vswitch at x% TX rate and measure throughput
phy2phy_scalability Scalability0PacketLoss for Phy2Phy
pvp_tput PacketLossRatio for PVP
pvp_back2back BackToBackFrames for PVP
pvvp_tput PacketLossRatio for PVVP
pvvp_back2back BackToBackFrames for PVVP
phy2phy_cpu_load CPU0PacketLoss for Phy2Phy
phy2phy_mem_load Same as CPU0PacketLoss but using a memory intensive app

Supported deployment scenarios:

  • Phy2Phy: Physical port -> vSwitch -> Physical port.
  • PVP: Physical port -> vSwitch -> VNF -> vSwitch -> Physical port.
  • PVVP: Physical port -> vSwitch -> VNF -> vSwitch -> VNF -> vSwitch -> Physical port.

Loopback applications in the Guest can be:

Supported traffic generators:

  • Ixia: IxOS and IxNet.
  • Spirent.
  • Dummy.

2. OPNFV Brahmaputra Results

The vsperf CI jobs that were used to obtain the results can be found at https://wiki.opnfv.org/wiki/vsperf_results.

The following table maps the results in the test dashboard to the appropriate test case in the VSPERF Framework and specifies the metric the vertical/Y axis is plotting. Please note, the presence of dpdk within a test name signifies that the vswitch under test was OVS with DPDK, while its absence indicates that the vswitch under test was stock OVS.

Dashboard Test Framework Test Metric Guest Interface
tput_ovsdpdk phy2phy_tput Throughput (FPS) N/A
tput_ovs phy2phy_tput Throughput (FPS) N/A
b2b_ovsdpdk back2back Back-to-back value N/A
b2b_ovs back2back Back-to-back value N/A
tput_mod_vlan_ovs phy2phy_tput_mod_vlan Throughput (FPS) N/A
tput_mod_vlan_ovsdpdk phy2phy_tput_mod_vlan Throughput (FPS) N/A
scalability_ovs phy2phy_scalability Throughput (FPS) N/A
scalability_ovsdpdk phy2phy_scalability Throughput (FPS) N/A
pvp_tput_ovsdpdkuser pvp_tput Throughput (FPS) vhost-user
pvp_tput_ovsvirtio pvp_tput Throughput (FPS) virtio-net
pvp_b2b_ovsdpdkuser pvp_back2back Back-to-back value vhost-user
pvp_b2b_ovsvirtio pvp_back2back Back-to-back value virtio-net
pvvp_tput_ovsdpdkuser pvvp_tput Throughput (FPS) vhost-user
pvvp_tput_ovsvirtio pvvp_tput Throughput (FPS) virtio-net
pvvp_b2b_ovsdpdkuser pvvp_back2back Throughput (FPS) vhost-user
pvvp_b2b_ovsvirtio pvvp_back2back Throughput (FPS) virtio-net

The loopback application in the VNF used for PVP and PVVP scenarios was DPDK testpmd.

3. Performance report for Open vSwitch with DPDK support

3.1. Introduction

The objective of the OPNFV project titled “Characterise vSwitch Performance for Telco NFV Use Cases”, is to evaluate a virtual switch to identify its suitability for a Telco Network Function Virtualization (NFV) environment. As well as this, the project aims to identify any gaps or bottlenecks in order to drive architectural changes to improve virtual switch performance and determinism. The purpose of this document is to summarize the results of the tests carried out on the virtual switch in the Network Function Virtualization Infrastructure (NFVI) and, from these results, provide evaluations and recommendations for the virtual switch. Test results will be outlined in details-of-LTR, preceded by the document-identifier and the scope and references).

This document is currently in draft form.

3.1.1. Document identifier

The document id will be used to uniquely identify versions of the LTR. The format for the document id will be: OPNFV_vswitchperf_LTR_rel_STATUS, the status is one of: DRAFT, REVIEWED, CORRECTED or FINAL. The document id for this version of the LTR is: OPNFV_vswitchperf_LTR_Brahmaputra_DRAFT.

3.1.2. Scope

The scope of this report is to detail the results of the tests that have been performed on the virtual switch. This report will also evaluate the results of these tests and, based on these evaluations, provide recommendations on the suitability of the virtual switch for use in a Telco NFV environment.

3.2. Details of the Level Test Report

This section provides a test-results-overview. Also included are the rationale and the conclusions.

3.2.1. Test ID: BACK2BACK

3.2.1.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 513153802b329c0584d72655460a4a264ad41e7c

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • DPDK Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: p2p
  • Traffic type: rfc2544
  • Bidirectional : True
3.2.1.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.1.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 26822
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:39:28
3.2.1.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 175985
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 13.22
majflt/s 0.00
VSZ 8057820
RSS 23261
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 175969
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52084
RSS 3460
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.1.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 5938555
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:39:28
3.2.1.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 175985
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 13.22
majflt/s 0.00
VSZ 8057820
RSS 23261
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 175969
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52084
RSS 3460
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.1.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 70488721
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:39:28
3.2.1.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 175985
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 13.22
majflt/s 0.00
VSZ 8057820
RSS 23261
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 175969
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52084
RSS 3460
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.1.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 35919540
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:39:28
3.2.1.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 175985
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 13.22
majflt/s 0.00
VSZ 8057820
RSS 23261
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 175969
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52084
RSS 3460
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.1.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 24382314
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:39:28
3.2.1.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 175985
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 13.22
majflt/s 0.00
VSZ 8057820
RSS 23261
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 175969
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52084
RSS 3460
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.7. Anomalies

No anomalies were detected during the course of this test.

3.2.1.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.2. Test ID: PVP_TPUT

3.2.2.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: 513153802b329c0584d72655460a4a264ad41e7c

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • DPDK Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

  • VNF: QemuDpdkVhostUser, Version: 2.5.0, GIT tag: a8c40fa2d667e585382080db36ac44e216b37a1c

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: pvp_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvp
  • Traffic type: rfc2544
  • Bidirectional : True
3.2.2.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.2.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4441037.260
tx_rate_mbps Unknown
throughput_rx_mbps 2273.811
tx_rate_percent 14.922
throughput_rx_percent 14.922
frame_loss_percent 0.000
min_latency_ns 4940.000
max_latency_ns 62800.000
avg_latency_ns 6407.000
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:32:10
guest_loopback_app testpmd
3.2.2.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 189641
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 16.03
majflt/s 0.00
VSZ 12256216
RSS 25387
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 189625
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52084
RSS 3496
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 189872
%usr 0.00
%system 0.34
%guest 200.23
%CPU 100.17
CPU
minflt/s 1.76
majflt/s 0.00
VSZ 2679184
RSS 218004
%MEM 0.33
kB_rd/s 0.00
kB_wr/s 7.83
kB_ccwr/s 0.00
3.2.2.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.2.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 5525507.976
tx_rate_mbps Unknown
throughput_rx_mbps 5658.120
tx_rate_percent 32.711
throughput_rx_percent 32.711
frame_loss_percent 0.000
min_latency_ns 6700.000
max_latency_ns 74860.000
avg_latency_ns 9282.000
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:32:10
guest_loopback_app testpmd
3.2.2.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 189641
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 16.03
majflt/s 0.00
VSZ 12256216
RSS 25387
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 189625
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52084
RSS 3496
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 189872
%usr 0.00
%system 0.34
%guest 200.23
%CPU 100.17
CPU
minflt/s 1.76
majflt/s 0.00
VSZ 2679184
RSS 218004
%MEM 0.33
kB_rd/s 0.00
kB_wr/s 7.83
kB_ccwr/s 0.00
3.2.2.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.2.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699074.345
tx_rate_mbps Unknown
throughput_rx_mbps 19247.409
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 50200.000
max_latency_ns 169660.000
avg_latency_ns 150576.000
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:32:10
guest_loopback_app testpmd
3.2.2.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 189641
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 16.03
majflt/s 0.00
VSZ 12256216
RSS 25387
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 189625
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52084
RSS 3496
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 189872
%usr 0.00
%system 0.34
%guest 200.23
%CPU 100.17
CPU
minflt/s 1.76
majflt/s 0.00
VSZ 2679184
RSS 218004
%MEM 0.33
kB_rd/s 0.00
kB_wr/s 7.83
kB_ccwr/s 0.00
3.2.2.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.2.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394574.934
tx_rate_mbps Unknown
throughput_rx_mbps 19616.358
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 53020.000
max_latency_ns 144360.000
avg_latency_ns 128020.500
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:32:10
guest_loopback_app testpmd
3.2.2.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 189641
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 16.03
majflt/s 0.00
VSZ 12256216
RSS 25387
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 189625
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52084
RSS 3496
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 189872
%usr 0.00
%system 0.34
%guest 200.23
%CPU 100.17
CPU
minflt/s 1.76
majflt/s 0.00
VSZ 2679184
RSS 218004
%MEM 0.33
kB_rd/s 0.00
kB_wr/s 7.83
kB_ccwr/s 0.00
3.2.2.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.2.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625455.626
tx_rate_mbps Unknown
throughput_rx_mbps 19739.533
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 47660.000
max_latency_ns 119840.000
avg_latency_ns 103841.000
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:32:10
guest_loopback_app testpmd
3.2.2.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 189641
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.43
CPU
minflt/s 16.03
majflt/s 0.00
VSZ 12256216
RSS 25387
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 189625
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52084
RSS 3496
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 189872
%usr 0.00
%system 0.34
%guest 200.23
%CPU 100.17
CPU
minflt/s 1.76
majflt/s 0.00
VSZ 2679184
RSS 218004
%MEM 0.33
kB_rd/s 0.00
kB_wr/s 7.83
kB_ccwr/s 0.00
3.2.2.7. Anomalies

No anomalies were detected during the course of this test.

3.2.2.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.3. Test ID: PHY2PHY_TPUT_MOD_VLAN

3.2.3.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 513153802b329c0584d72655460a4a264ad41e7c

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • DPDK Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: phy2phy_tput_mod_vlan
  • Description: LTD.Throughput.RFC2544.PacketLossRatioFrameModification
  • Deployment: p2p
  • Traffic type: rfc2544
  • Bidirectional : False
3.2.3.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.3.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 12118686.191
tx_rate_mbps Unknown
throughput_rx_mbps 6592.565
tx_rate_percent 81.438
throughput_rx_percent 85.316
frame_loss_percent 0.000
min_latency_ns 4860.000
max_latency_ns 167060.000
avg_latency_ns 8773.000
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:35:19
3.2.3.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 181152
%usr 200.32
%system 0.12
%guest 0.00
%CPU 200.44
CPU
minflt/s 22.33
majflt/s 0.00
VSZ 8057816
RSS 25268
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 181136
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52088
RSS 3460
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.3.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.3.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 7792714.757
tx_rate_mbps Unknown
throughput_rx_mbps 8229.107
tx_rate_percent 92.266
throughput_rx_percent 94.759
frame_loss_percent 0.000
min_latency_ns 4900.000
max_latency_ns 98460.000
avg_latency_ns 5756.000
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:35:19
3.2.3.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 181152
%usr 200.32
%system 0.12
%guest 0.00
%CPU 200.44
CPU
minflt/s 22.33
majflt/s 0.00
VSZ 8057816
RSS 25268
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 181136
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52088
RSS 3460
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.3.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.3.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2331451.900
tx_rate_mbps Unknown
throughput_rx_mbps 9624.233
tx_rate_percent 99.227
throughput_rx_percent 99.973
frame_loss_percent 0.000
min_latency_ns 6080.000
max_latency_ns 39400.000
avg_latency_ns 14273.000
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:35:19
3.2.3.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 181152
%usr 200.32
%system 0.12
%guest 0.00
%CPU 200.44
CPU
minflt/s 22.33
majflt/s 0.00
VSZ 8057816
RSS 25268
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 181136
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52088
RSS 3460
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.3.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.3.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1188058.140
tx_rate_mbps Unknown
throughput_rx_mbps 9770.590
tx_rate_percent 99.227
throughput_rx_percent 99.607
frame_loss_percent 0.000
min_latency_ns 5800.000
max_latency_ns 44040.000
avg_latency_ns 7701.000
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:35:19
3.2.3.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 181152
%usr 200.32
%system 0.12
%guest 0.00
%CPU 200.44
CPU
minflt/s 22.33
majflt/s 0.00
VSZ 8057816
RSS 25268
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 181136
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52088
RSS 3460
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.3.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.3.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 806458.196
tx_rate_mbps Unknown
throughput_rx_mbps 9819.435
tx_rate_percent 99.227
throughput_rx_percent 99.485
frame_loss_percent 0.000
min_latency_ns 6220.000
max_latency_ns 34160.000
avg_latency_ns 7641.000
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:35:19
3.2.3.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 181152
%usr 200.32
%system 0.12
%guest 0.00
%CPU 200.44
CPU
minflt/s 22.33
majflt/s 0.00
VSZ 8057816
RSS 25268
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 181136
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52088
RSS 3460
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.3.7. Anomalies

No anomalies were detected during the course of this test.

3.2.3.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.4. Test ID: PVVP_BACK2BACK

3.2.4.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: pvvp

  • vswitchperf: GIT tag: 513153802b329c0584d72655460a4a264ad41e7c

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • DPDK Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

  • VNF: QemuDpdkVhostUser, Version: 2.5.0, GIT tag: a8c40fa2d667e585382080db36ac44e216b37a1c

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: pvvp_back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: pvvp
  • Traffic type: rfc2544
  • Bidirectional : True
3.2.4.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.4.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1702
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:55:19
guest_loopback_app testpmd testpmd
3.2.4.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 10475
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 14.72
majflt/s 0.00
VSZ 16454620
RSS 25561
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 10459
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52076
RSS 3456
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 10859
%usr 0.00
%system 0.26
%guest 200.35
%CPU 100.17
CPU
minflt/s 1.59
majflt/s 0.00
VSZ 2657684
RSS 292039
%MEM 0.44
kB_rd/s 0.00
kB_wr/s 4.76
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 10721
%usr 0.00
%system 0.16
%guest 200.38
%CPU 100.16
CPU
minflt/s 1.30
majflt/s 0.00
VSZ 2645380
RSS 311444
%MEM 0.47
kB_rd/s 0.00
kB_wr/s 0.04
kB_ccwr/s 0.00
3.2.4.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.4.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 3866
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:55:19
guest_loopback_app testpmd testpmd
3.2.4.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 10475
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 14.72
majflt/s 0.00
VSZ 16454620
RSS 25561
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 10459
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52076
RSS 3456
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 10859
%usr 0.00
%system 0.26
%guest 200.35
%CPU 100.17
CPU
minflt/s 1.59
majflt/s 0.00
VSZ 2657684
RSS 292039
%MEM 0.44
kB_rd/s 0.00
kB_wr/s 4.76
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 10721
%usr 0.00
%system 0.16
%guest 200.38
%CPU 100.16
CPU
minflt/s 1.30
majflt/s 0.00
VSZ 2645380
RSS 311444
%MEM 0.47
kB_rd/s 0.00
kB_wr/s 0.04
kB_ccwr/s 0.00
3.2.4.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.4.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1075
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:55:19
guest_loopback_app testpmd testpmd
3.2.4.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 10475
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 14.72
majflt/s 0.00
VSZ 16454620
RSS 25561
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 10459
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52076
RSS 3456
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 10859
%usr 0.00
%system 0.26
%guest 200.35
%CPU 100.17
CPU
minflt/s 1.59
majflt/s 0.00
VSZ 2657684
RSS 292039
%MEM 0.44
kB_rd/s 0.00
kB_wr/s 4.76
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 10721
%usr 0.00
%system 0.16
%guest 200.38
%CPU 100.16
CPU
minflt/s 1.30
majflt/s 0.00
VSZ 2645380
RSS 311444
%MEM 0.47
kB_rd/s 0.00
kB_wr/s 0.04
kB_ccwr/s 0.00
3.2.4.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.4.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 11224856
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:55:19
guest_loopback_app testpmd testpmd
3.2.4.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 10475
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 14.72
majflt/s 0.00
VSZ 16454620
RSS 25561
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 10459
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52076
RSS 3456
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 10859
%usr 0.00
%system 0.26
%guest 200.35
%CPU 100.17
CPU
minflt/s 1.59
majflt/s 0.00
VSZ 2657684
RSS 292039
%MEM 0.44
kB_rd/s 0.00
kB_wr/s 4.76
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 10721
%usr 0.00
%system 0.16
%guest 200.38
%CPU 100.16
CPU
minflt/s 1.30
majflt/s 0.00
VSZ 2645380
RSS 311444
%MEM 0.47
kB_rd/s 0.00
kB_wr/s 0.04
kB_ccwr/s 0.00
3.2.4.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.4.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 14366129
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:55:19
guest_loopback_app testpmd testpmd
3.2.4.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 10475
%usr 200.33
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 14.72
majflt/s 0.00
VSZ 16454620
RSS 25561
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 10459
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52076
RSS 3456
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 10859
%usr 0.00
%system 0.26
%guest 200.35
%CPU 100.17
CPU
minflt/s 1.59
majflt/s 0.00
VSZ 2657684
RSS 292039
%MEM 0.44
kB_rd/s 0.00
kB_wr/s 4.76
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 10721
%usr 0.00
%system 0.16
%guest 200.38
%CPU 100.16
CPU
minflt/s 1.30
majflt/s 0.00
VSZ 2645380
RSS 311444
%MEM 0.47
kB_rd/s 0.00
kB_wr/s 0.04
kB_ccwr/s 0.00
3.2.4.7. Anomalies

No anomalies were detected during the course of this test.

3.2.4.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.5. Test ID: PHY2PHY_TPUT

3.2.5.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 513153802b329c0584d72655460a4a264ad41e7c

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • DPDK Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: phy2phy_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: p2p
  • Traffic type: rfc2544
  • Bidirectional : True
3.2.5.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.5.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 23086559.686
tx_rate_mbps Unknown
throughput_rx_mbps 11820.319
tx_rate_percent 77.570
throughput_rx_percent 77.570
frame_loss_percent 0.000
min_latency_ns 6840.000
max_latency_ns 336900.000
avg_latency_ns 8335.000
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:26:50
3.2.5.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 172257
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 6.61
majflt/s 0.00
VSZ 8057816
RSS 25359
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 172241
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52088
RSS 3460
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.5.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.5.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 16238708.639
tx_rate_mbps Unknown
throughput_rx_mbps 16628.438
tx_rate_percent 96.133
throughput_rx_percent 96.133
frame_loss_percent 0.000
min_latency_ns 8060.000
max_latency_ns 327160.000
avg_latency_ns 9770.500
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:26:50
3.2.5.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 172257
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 6.61
majflt/s 0.00
VSZ 8057816
RSS 25359
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 172241
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52088
RSS 3460
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.5.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.5.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699196.911
tx_rate_mbps Unknown
throughput_rx_mbps 19247.911
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 22320.000
max_latency_ns 55720.000
avg_latency_ns 44299.000
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:26:50
3.2.5.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 172257
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 6.61
majflt/s 0.00
VSZ 8057816
RSS 25359
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 172241
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52088
RSS 3460
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.5.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.5.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394611.252
tx_rate_mbps Unknown
throughput_rx_mbps 19616.655
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 35560.000
max_latency_ns 67340.000
avg_latency_ns 56068.000
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:26:50
3.2.5.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 172257
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 6.61
majflt/s 0.00
VSZ 8057816
RSS 25359
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 172241
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52088
RSS 3460
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.5.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.5.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625472.249
tx_rate_mbps Unknown
throughput_rx_mbps 19739.735
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 37620.000
max_latency_ns 66860.000
avg_latency_ns 55655.000
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:26:50
3.2.5.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 172257
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 6.61
majflt/s 0.00
VSZ 8057816
RSS 25359
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 172241
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52088
RSS 3460
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.5.7. Anomalies

No anomalies were detected during the course of this test.

3.2.5.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.6. Test ID: PHY2PHY_SCALABILITY

3.2.6.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 513153802b329c0584d72655460a4a264ad41e7c

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • DPDK Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: phy2phy_scalability
  • Description: LTD.Scalability.Flows.RFC2544.0PacketLoss
  • Deployment: p2p
  • Traffic type: rfc2544
  • Bidirectional : True
3.2.6.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.6.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 23316602.374
tx_rate_mbps Unknown
throughput_rx_mbps 11938.100
tx_rate_percent 78.344
throughput_rx_percent 78.344
frame_loss_percent 0.000
min_latency_ns 4840.000
max_latency_ns 112600.000
avg_latency_ns 9074.500
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:27:47
stream_count 8000
match_type L4
pre-installed_flows No
3.2.6.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 185904
%usr 200.32
%system 0.12
%guest 0.00
%CPU 200.43
CPU
minflt/s 17.24
majflt/s 0.00
VSZ 8057816
RSS 25224
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 185888
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52084
RSS 3460
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.6.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.6.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 15585429.888
tx_rate_mbps Unknown
throughput_rx_mbps 15959.480
tx_rate_percent 92.266
throughput_rx_percent 92.266
frame_loss_percent 0.000
min_latency_ns 7580.000
max_latency_ns 57680.000
avg_latency_ns 8534.000
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:27:47
stream_count 8000
match_type L4
pre-installed_flows No
3.2.6.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 185904
%usr 200.32
%system 0.12
%guest 0.00
%CPU 200.43
CPU
minflt/s 17.24
majflt/s 0.00
VSZ 8057816
RSS 25224
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 185888
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52084
RSS 3460
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.6.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.6.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699192.760
tx_rate_mbps Unknown
throughput_rx_mbps 19247.894
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 31280.000
max_latency_ns 67060.000
avg_latency_ns 56214.500
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:27:47
stream_count 8000
match_type L4
pre-installed_flows No
3.2.6.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 185904
%usr 200.32
%system 0.12
%guest 0.00
%CPU 200.43
CPU
minflt/s 17.24
majflt/s 0.00
VSZ 8057816
RSS 25224
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 185888
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52084
RSS 3460
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.6.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.6.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394604.172
tx_rate_mbps Unknown
throughput_rx_mbps 19616.597
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 47580.000
max_latency_ns 91700.000
avg_latency_ns 77829.000
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:27:47
stream_count 8000
match_type L4
pre-installed_flows No
3.2.6.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 185904
%usr 200.32
%system 0.12
%guest 0.00
%CPU 200.43
CPU
minflt/s 17.24
majflt/s 0.00
VSZ 8057816
RSS 25224
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 185888
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52084
RSS 3460
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.6.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.6.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625467.871
tx_rate_mbps Unknown
throughput_rx_mbps 19739.682
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 41680.000
max_latency_ns 78980.000
avg_latency_ns 67925.000
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:27:47
stream_count 8000
match_type L4
pre-installed_flows No
3.2.6.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 185904
%usr 200.32
%system 0.12
%guest 0.00
%CPU 200.43
CPU
minflt/s 17.24
majflt/s 0.00
VSZ 8057816
RSS 25224
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 185888
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52084
RSS 3460
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.6.7. Anomalies

No anomalies were detected during the course of this test.

3.2.6.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.7. Test ID: PVP_BACK2BACK

3.2.7.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: 513153802b329c0584d72655460a4a264ad41e7c

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • DPDK Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

  • VNF: QemuDpdkVhostUser, Version: 2.5.0, GIT tag: a8c40fa2d667e585382080db36ac44e216b37a1c

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: pvp_back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: pvp
  • Traffic type: rfc2544
  • Bidirectional : True
3.2.7.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.7.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1277
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:55:16
guest_loopback_app testpmd
3.2.7.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 194059
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 14.72
majflt/s 0.00
VSZ 12256216
RSS 27372
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 194043
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52080
RSS 3452
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 194290
%usr 0.00
%system 0.03
%guest 200.12
%CPU 100.16
CPU
minflt/s 1.63
majflt/s 0.00
VSZ 2673024
RSS 298749
%MEM 0.45
kB_rd/s 0.00
kB_wr/s 4.38
kB_ccwr/s 0.00
3.2.7.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.7.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1449
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:55:16
guest_loopback_app testpmd
3.2.7.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 194059
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 14.72
majflt/s 0.00
VSZ 12256216
RSS 27372
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 194043
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52080
RSS 3452
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 194290
%usr 0.00
%system 0.03
%guest 200.12
%CPU 100.16
CPU
minflt/s 1.63
majflt/s 0.00
VSZ 2673024
RSS 298749
%MEM 0.45
kB_rd/s 0.00
kB_wr/s 4.38
kB_ccwr/s 0.00
3.2.7.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.7.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 40345263
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:55:16
guest_loopback_app testpmd
3.2.7.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 194059
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 14.72
majflt/s 0.00
VSZ 12256216
RSS 27372
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 194043
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52080
RSS 3452
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 194290
%usr 0.00
%system 0.03
%guest 200.12
%CPU 100.16
CPU
minflt/s 1.63
majflt/s 0.00
VSZ 2673024
RSS 298749
%MEM 0.45
kB_rd/s 0.00
kB_wr/s 4.38
kB_ccwr/s 0.00
3.2.7.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.7.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 35919540
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:55:16
guest_loopback_app testpmd
3.2.7.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 194059
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 14.72
majflt/s 0.00
VSZ 12256216
RSS 27372
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 194043
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52080
RSS 3452
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 194290
%usr 0.00
%system 0.03
%guest 200.12
%CPU 100.16
CPU
minflt/s 1.63
majflt/s 0.00
VSZ 2673024
RSS 298749
%MEM 0.45
kB_rd/s 0.00
kB_wr/s 4.38
kB_ccwr/s 0.00
3.2.7.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.7.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 24382314
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:55:16
guest_loopback_app testpmd
3.2.7.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 194059
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 14.72
majflt/s 0.00
VSZ 12256216
RSS 27372
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 194043
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52080
RSS 3452
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 194290
%usr 0.00
%system 0.03
%guest 200.12
%CPU 100.16
CPU
minflt/s 1.63
majflt/s 0.00
VSZ 2673024
RSS 298749
%MEM 0.45
kB_rd/s 0.00
kB_wr/s 4.38
kB_ccwr/s 0.00
3.2.7.7. Anomalies

No anomalies were detected during the course of this test.

3.2.7.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.8. Test ID: PVVP_TPUT

3.2.8.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: pvvp

  • vswitchperf: GIT tag: 513153802b329c0584d72655460a4a264ad41e7c

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • DPDK Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

  • VNF: QemuDpdkVhostUser, Version: 2.5.0, GIT tag: a8c40fa2d667e585382080db36ac44e216b37a1c

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: pvvp_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvvp
  • Traffic type: rfc2544
  • Bidirectional : True
3.2.8.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.8.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2369329.404
tx_rate_mbps Unknown
throughput_rx_mbps 1213.097
tx_rate_percent 7.961
throughput_rx_percent 7.961
frame_loss_percent 0.000
min_latency_ns 6600.000
max_latency_ns 71200.000
avg_latency_ns 9595.500
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:36:28
guest_loopback_app testpmd testpmd
3.2.8.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 5403
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 15.88
majflt/s 0.00
VSZ 16454616
RSS 23426
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 5387
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52092
RSS 3456
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 5771
%usr 0.00
%system 0.20
%guest 200.20
%CPU 100.16
CPU
minflt/s 1.67
majflt/s 0.00
VSZ 2638224
RSS 190196
%MEM 0.29
kB_rd/s 0.00
kB_wr/s 6.46
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 5645
%usr 0.00
%system 0.03
%guest 200.11
%CPU 100.15
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2651536
RSS 237538
%MEM 0.36
kB_rd/s 0.00
kB_wr/s 0.14
kB_ccwr/s 0.00
3.2.8.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.8.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2781887.433
tx_rate_mbps Unknown
throughput_rx_mbps 2848.653
tx_rate_percent 16.469
throughput_rx_percent 16.469
frame_loss_percent 0.000
min_latency_ns 8180.000
max_latency_ns 78720.000
avg_latency_ns 11641.500
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:36:28
guest_loopback_app testpmd testpmd
3.2.8.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 5403
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 15.88
majflt/s 0.00
VSZ 16454616
RSS 23426
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 5387
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52092
RSS 3456
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 5771
%usr 0.00
%system 0.20
%guest 200.20
%CPU 100.16
CPU
minflt/s 1.67
majflt/s 0.00
VSZ 2638224
RSS 190196
%MEM 0.29
kB_rd/s 0.00
kB_wr/s 6.46
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 5645
%usr 0.00
%system 0.03
%guest 200.11
%CPU 100.15
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2651536
RSS 237538
%MEM 0.36
kB_rd/s 0.00
kB_wr/s 0.14
kB_ccwr/s 0.00
3.2.8.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.8.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 3536168.564
tx_rate_mbps Unknown
throughput_rx_mbps 14484.146
tx_rate_percent 75.250
throughput_rx_percent 75.250
frame_loss_percent 0.000
min_latency_ns 13860.000
max_latency_ns 128800.000
avg_latency_ns 33809.500
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:36:28
guest_loopback_app testpmd testpmd
3.2.8.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 5403
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 15.88
majflt/s 0.00
VSZ 16454616
RSS 23426
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 5387
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52092
RSS 3456
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 5771
%usr 0.00
%system 0.20
%guest 200.20
%CPU 100.16
CPU
minflt/s 1.67
majflt/s 0.00
VSZ 2638224
RSS 190196
%MEM 0.29
kB_rd/s 0.00
kB_wr/s 6.46
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 5645
%usr 0.00
%system 0.03
%guest 200.11
%CPU 100.15
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2651536
RSS 237538
%MEM 0.36
kB_rd/s 0.00
kB_wr/s 0.14
kB_ccwr/s 0.00
3.2.8.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.8.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1672317.175
tx_rate_mbps Unknown
throughput_rx_mbps 13699.622
tx_rate_percent 69.836
throughput_rx_percent 69.836
frame_loss_percent 0.000
min_latency_ns 10300.000
max_latency_ns 205420.000
avg_latency_ns 15718.000
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:36:28
guest_loopback_app testpmd testpmd
3.2.8.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 5403
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 15.88
majflt/s 0.00
VSZ 16454616
RSS 23426
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 5387
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52092
RSS 3456
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 5771
%usr 0.00
%system 0.20
%guest 200.20
%CPU 100.16
CPU
minflt/s 1.67
majflt/s 0.00
VSZ 2638224
RSS 190196
%MEM 0.29
kB_rd/s 0.00
kB_wr/s 6.46
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 5645
%usr 0.00
%system 0.03
%guest 200.11
%CPU 100.15
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2651536
RSS 237538
%MEM 0.36
kB_rd/s 0.00
kB_wr/s 0.14
kB_ccwr/s 0.00
3.2.8.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.8.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1424332.709
tx_rate_mbps Unknown
throughput_rx_mbps 17297.096
tx_rate_percent 87.625
throughput_rx_percent 87.625
frame_loss_percent 0.000
min_latency_ns 12500.000
max_latency_ns 77960.000
avg_latency_ns 18080.500
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:36:28
guest_loopback_app testpmd testpmd
3.2.8.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 5403
%usr 200.32
%system 0.11
%guest 0.00
%CPU 200.44
CPU
minflt/s 15.88
majflt/s 0.00
VSZ 16454616
RSS 23426
%MEM 0.04
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 5387
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 52092
RSS 3456
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 5771
%usr 0.00
%system 0.20
%guest 200.20
%CPU 100.16
CPU
minflt/s 1.67
majflt/s 0.00
VSZ 2638224
RSS 190196
%MEM 0.29
kB_rd/s 0.00
kB_wr/s 6.46
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 5645
%usr 0.00
%system 0.03
%guest 200.11
%CPU 100.15
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2651536
RSS 237538
%MEM 0.36
kB_rd/s 0.00
kB_wr/s 0.14
kB_ccwr/s 0.00
3.2.8.7. Anomalies

No anomalies were detected during the course of this test.

3.2.8.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.3. Rationale for decisions

The tests conducted do not have pass/fail/conditional-pass criteria. The test is simply conducted and the results are reported.

3.4. Conclusions and recommendations

The test results are stable. The vsperf CI jobs that were used to obtain the results can be found at https://artifacts.opnfv.org/logs/vswitchperf/intel-pod3/2016-12-02_10-35-07/vswitchperf_logs_2016-12-02_10-35-07.tar.gz.

3.5. General

3.5.1. Glossary

  • NFV - Network Function Virtualization
  • Mbps - 1,000,000bps

3.5.2. Document change procedures and history

Document ID Author Date Modified
OPNFV_vswitchperf_LTR_ver_1.0_Jan_15_CN_DRAFT Christopher Nolan 23/01/2015
OPNFV_vswitchperf_LTR_ver_1.1_Jan_15_CN_DRAFT Christopher Nolan 28/01/2015

4. Performance report for Open vSwitch

4.1. Introduction

The objective of the OPNFV project titled “Characterise vSwitch Performance for Telco NFV Use Cases”, is to evaluate a virtual switch to identify its suitability for a Telco Network Function Virtualization (NFV) environment. As well as this, the project aims to identify any gaps or bottlenecks in order to drive architectural changes to improve virtual switch performance and determinism. The purpose of this document is to summarize the results of the tests carried out on the virtual switch in the Network Function Virtualization Infrastructure (NFVI) and, from these results, provide evaluations and recommendations for the virtual switch. Test results will be outlined in details-of-LTR, preceded by the document-identifier and the scope and references).

This document is currently in draft form.

4.1.1. Document identifier

The document id will be used to uniquely identify versions of the LTR. The format for the document id will be: OPNFV_vswitchperf_LTR_rel_STATUS, the status is one of: DRAFT, REVIEWED, CORRECTED or FINAL. The document id for this version of the LTR is: OPNFV_vswitchperf_LTR_Brahmaputra_DRAFT.

4.1.2. Scope

The scope of this report is to detail the results of the tests that have been performed on the virtual switch. This report will also evaluate the results of these tests and, based on these evaluations, provide recommendations on the suitability of the virtual switch for use in a Telco NFV environment.

4.2. Details of the Level Test Report

This section provides a test-results-overview. Also included are the rationale and the conclusions.

4.2.1. Test ID: BACK2BACK

4.2.1.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 513153802b329c0584d72655460a4a264ad41e7c

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

Below are test details:

  • Test ID: back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: p2p
  • Traffic type: rfc2544
  • Bidirectional : True
4.2.1.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.1.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 5108
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:46:06
4.2.1.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 22104
%usr 0.36
%system 4.44
%guest 0.00
%CPU 4.80
CPU
minflt/s 5.53
majflt/s 0.00
VSZ 6609928
RSS 32079
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 22101
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.1.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 3382
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:46:06
4.2.1.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 22104
%usr 0.36
%system 4.44
%guest 0.00
%CPU 4.80
CPU
minflt/s 5.53
majflt/s 0.00
VSZ 6609928
RSS 32079
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 22101
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.1.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1344
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:46:06
4.2.1.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 22104
%usr 0.36
%system 4.44
%guest 0.00
%CPU 4.80
CPU
minflt/s 5.53
majflt/s 0.00
VSZ 6609928
RSS 32079
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 22101
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.1.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1096
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:46:06
4.2.1.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 22104
%usr 0.36
%system 4.44
%guest 0.00
%CPU 4.80
CPU
minflt/s 5.53
majflt/s 0.00
VSZ 6609928
RSS 32079
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 22101
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.1.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1116
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:46:06
4.2.1.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 22104
%usr 0.36
%system 4.44
%guest 0.00
%CPU 4.80
CPU
minflt/s 5.53
majflt/s 0.00
VSZ 6609928
RSS 32079
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 22101
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.7. Anomalies

No anomalies were detected during the course of this test.

4.2.1.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.2. Test ID: PVP_TPUT

4.2.2.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: 513153802b329c0584d72655460a4a264ad41e7c

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • VNF: QemuVirtioNet, Version: 2.5.0, GIT tag: a8c40fa2d667e585382080db36ac44e216b37a1c

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: pvp_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvp
  • Traffic type: rfc2544
  • Bidirectional : True
4.2.2.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.2.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 525745.222
tx_rate_mbps Unknown
throughput_rx_mbps 269.182
tx_rate_percent 1.773
throughput_rx_percent 1.767
frame_loss_percent 0.515
min_latency_ns 10380.000
max_latency_ns 73381200.000
avg_latency_ns 102015.000
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:35:01
guest_loopback_app testpmd
4.2.2.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 36505
%usr 0.35
%system 4.33
%guest 0.00
%CPU 4.68
CPU
minflt/s 5.84
majflt/s 0.00
VSZ 6609936
RSS 47725
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 36502
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45904
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 36791
%usr 0.00
%system 0.87
%guest 199.71
%CPU 100.17
CPU
minflt/s 1.69
majflt/s 0.00
VSZ 2658824
RSS 215145
%MEM 0.33
kB_rd/s 0.00
kB_wr/s 7.06
kB_ccwr/s 0.00
4.2.2.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.2.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 299537.458
tx_rate_mbps Unknown
throughput_rx_mbps 306.726
tx_rate_percent 1.773
throughput_rx_percent 1.773
frame_loss_percent 0.011
min_latency_ns 8520.000
max_latency_ns 7906340.000
avg_latency_ns 45917.500
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:35:01
guest_loopback_app testpmd
4.2.2.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 36505
%usr 0.35
%system 4.33
%guest 0.00
%CPU 4.68
CPU
minflt/s 5.84
majflt/s 0.00
VSZ 6609936
RSS 47725
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 36502
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45904
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 36791
%usr 0.00
%system 0.87
%guest 199.71
%CPU 100.17
CPU
minflt/s 1.69
majflt/s 0.00
VSZ 2658824
RSS 215145
%MEM 0.33
kB_rd/s 0.00
kB_wr/s 7.06
kB_ccwr/s 0.00
4.2.2.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.2.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 192375.620
tx_rate_mbps Unknown
throughput_rx_mbps 787.971
tx_rate_percent 4.094
throughput_rx_percent 4.094
frame_loss_percent 0.000
min_latency_ns 15240.000
max_latency_ns 2181740.000
avg_latency_ns 102197.000
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:35:01
guest_loopback_app testpmd
4.2.2.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 36505
%usr 0.35
%system 4.33
%guest 0.00
%CPU 4.68
CPU
minflt/s 5.84
majflt/s 0.00
VSZ 6609936
RSS 47725
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 36502
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45904
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 36791
%usr 0.00
%system 0.87
%guest 199.71
%CPU 100.17
CPU
minflt/s 1.69
majflt/s 0.00
VSZ 2658824
RSS 215145
%MEM 0.33
kB_rd/s 0.00
kB_wr/s 7.06
kB_ccwr/s 0.00
4.2.2.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.2.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 172115.085
tx_rate_mbps Unknown
throughput_rx_mbps 1409.967
tx_rate_percent 7.188
throughput_rx_percent 7.188
frame_loss_percent 0.000
min_latency_ns 13800.000
max_latency_ns 985980.000
avg_latency_ns 99281.500
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:35:01
guest_loopback_app testpmd
4.2.2.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 36505
%usr 0.35
%system 4.33
%guest 0.00
%CPU 4.68
CPU
minflt/s 5.84
majflt/s 0.00
VSZ 6609936
RSS 47725
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 36502
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45904
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 36791
%usr 0.00
%system 0.87
%guest 199.71
%CPU 100.17
CPU
minflt/s 1.69
majflt/s 0.00
VSZ 2658824
RSS 215145
%MEM 0.33
kB_rd/s 0.00
kB_wr/s 7.06
kB_ccwr/s 0.00
4.2.2.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.2.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 217411.001
tx_rate_mbps Unknown
throughput_rx_mbps 2640.239
tx_rate_percent 13.375
throughput_rx_percent 13.375
frame_loss_percent 0.000
min_latency_ns 26780.000
max_latency_ns 832500.000
avg_latency_ns 108524.000
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:35:01
guest_loopback_app testpmd
4.2.2.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 36505
%usr 0.35
%system 4.33
%guest 0.00
%CPU 4.68
CPU
minflt/s 5.84
majflt/s 0.00
VSZ 6609936
RSS 47725
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 36502
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45904
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 36791
%usr 0.00
%system 0.87
%guest 199.71
%CPU 100.17
CPU
minflt/s 1.69
majflt/s 0.00
VSZ 2658824
RSS 215145
%MEM 0.33
kB_rd/s 0.00
kB_wr/s 7.06
kB_ccwr/s 0.00
4.2.2.7. Anomalies

No anomalies were detected during the course of this test.

4.2.2.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.3. Test ID: PHY2PHY_TPUT_MOD_VLAN

4.2.3.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 513153802b329c0584d72655460a4a264ad41e7c

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

Below are test details:

  • Test ID: phy2phy_tput_mod_vlan
  • Description: LTD.Throughput.RFC2544.PacketLossRatioFrameModification
  • Deployment: p2p
  • Traffic type: rfc2544
  • Bidirectional : False
4.2.3.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.3.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 724284.609
tx_rate_mbps Unknown
throughput_rx_mbps 394.011
tx_rate_percent 4.867
throughput_rx_percent 5.099
frame_loss_percent 0.000
min_latency_ns 4080.000
max_latency_ns 5119400.000
avg_latency_ns 542697.000
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:33:41
4.2.3.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 27834
%usr 0.35
%system 4.43
%guest 0.00
%CPU 4.78
CPU
minflt/s 5.27
majflt/s 0.00
VSZ 6609932
RSS 32164
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 27831
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.3.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.3.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 672337.337
tx_rate_mbps Unknown
throughput_rx_mbps 709.988
tx_rate_percent 7.961
throughput_rx_percent 8.176
frame_loss_percent 0.000
min_latency_ns 4460.000
max_latency_ns 3521320.000
avg_latency_ns 193103.000
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:33:41
4.2.3.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 27834
%usr 0.35
%system 4.43
%guest 0.00
%CPU 4.78
CPU
minflt/s 5.27
majflt/s 0.00
VSZ 6609932
RSS 32164
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 27831
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.3.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.3.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 586856.086
tx_rate_mbps Unknown
throughput_rx_mbps 2422.542
tx_rate_percent 24.977
throughput_rx_percent 25.164
frame_loss_percent 0.000
min_latency_ns 5420.000
max_latency_ns 1726080.000
avg_latency_ns 107831.000
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:33:41
4.2.3.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 27834
%usr 0.35
%system 4.43
%guest 0.00
%CPU 4.78
CPU
minflt/s 5.27
majflt/s 0.00
VSZ 6609932
RSS 32164
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 27831
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.3.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.3.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 576865.047
tx_rate_mbps Unknown
throughput_rx_mbps 4744.138
tx_rate_percent 48.180
throughput_rx_percent 48.364
frame_loss_percent 0.000
min_latency_ns 5900.000
max_latency_ns 1522080.000
avg_latency_ns 93987.000
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:33:41
4.2.3.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 27834
%usr 0.35
%system 4.43
%guest 0.00
%CPU 4.78
CPU
minflt/s 5.27
majflt/s 0.00
VSZ 6609932
RSS 32164
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 27831
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.3.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.3.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 567588.524
tx_rate_mbps Unknown
throughput_rx_mbps 6910.958
tx_rate_percent 69.836
throughput_rx_percent 70.018
frame_loss_percent 0.000
min_latency_ns 6440.000
max_latency_ns 1026560.000
avg_latency_ns 84680.000
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:33:41
4.2.3.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 27834
%usr 0.35
%system 4.43
%guest 0.00
%CPU 4.78
CPU
minflt/s 5.27
majflt/s 0.00
VSZ 6609932
RSS 32164
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 27831
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.3.7. Anomalies

No anomalies were detected during the course of this test.

4.2.3.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.4. Test ID: PVVP_BACK2BACK

4.2.4.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: pvvp

  • vswitchperf: GIT tag: 513153802b329c0584d72655460a4a264ad41e7c

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • VNF: QemuVirtioNet, Version: 2.5.0, GIT tag: a8c40fa2d667e585382080db36ac44e216b37a1c

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: pvvp_back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: pvvp
  • Traffic type: rfc2544
  • Bidirectional : True
4.2.4.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.4.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:46:47
guest_loopback_app testpmd testpmd
4.2.4.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 52014
%usr 0.33
%system 4.30
%guest 0.00
%CPU 4.63
CPU
minflt/s 5.78
majflt/s 0.00
VSZ 6610056
RSS 42081
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 52011
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 52389
%usr 0.00
%system 0.08
%guest 200.07
%CPU 100.16
CPU
minflt/s 1.68
majflt/s 0.00
VSZ 2629997
RSS 220554
%MEM 0.34
kB_rd/s 0.00
kB_wr/s 5.44
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 52365
%usr 0.00
%system 0.03
%guest 200.07
%CPU 100.16
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2657816
RSS 269403
%MEM 0.41
kB_rd/s 0.00
kB_wr/s 0.02
kB_ccwr/s 0.00
4.2.4.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.4.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:46:47
guest_loopback_app testpmd testpmd
4.2.4.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 52014
%usr 0.33
%system 4.30
%guest 0.00
%CPU 4.63
CPU
minflt/s 5.78
majflt/s 0.00
VSZ 6610056
RSS 42081
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 52011
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 52389
%usr 0.00
%system 0.08
%guest 200.07
%CPU 100.16
CPU
minflt/s 1.68
majflt/s 0.00
VSZ 2629997
RSS 220554
%MEM 0.34
kB_rd/s 0.00
kB_wr/s 5.44
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 52365
%usr 0.00
%system 0.03
%guest 200.07
%CPU 100.16
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2657816
RSS 269403
%MEM 0.41
kB_rd/s 0.00
kB_wr/s 0.02
kB_ccwr/s 0.00
4.2.4.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.4.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 268
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:46:47
guest_loopback_app testpmd testpmd
4.2.4.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 52014
%usr 0.33
%system 4.30
%guest 0.00
%CPU 4.63
CPU
minflt/s 5.78
majflt/s 0.00
VSZ 6610056
RSS 42081
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 52011
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 52389
%usr 0.00
%system 0.08
%guest 200.07
%CPU 100.16
CPU
minflt/s 1.68
majflt/s 0.00
VSZ 2629997
RSS 220554
%MEM 0.34
kB_rd/s 0.00
kB_wr/s 5.44
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 52365
%usr 0.00
%system 0.03
%guest 200.07
%CPU 100.16
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2657816
RSS 269403
%MEM 0.41
kB_rd/s 0.00
kB_wr/s 0.02
kB_ccwr/s 0.00
4.2.4.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.4.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:46:47
guest_loopback_app testpmd testpmd
4.2.4.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 52014
%usr 0.33
%system 4.30
%guest 0.00
%CPU 4.63
CPU
minflt/s 5.78
majflt/s 0.00
VSZ 6610056
RSS 42081
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 52011
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 52389
%usr 0.00
%system 0.08
%guest 200.07
%CPU 100.16
CPU
minflt/s 1.68
majflt/s 0.00
VSZ 2629997
RSS 220554
%MEM 0.34
kB_rd/s 0.00
kB_wr/s 5.44
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 52365
%usr 0.00
%system 0.03
%guest 200.07
%CPU 100.16
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2657816
RSS 269403
%MEM 0.41
kB_rd/s 0.00
kB_wr/s 0.02
kB_ccwr/s 0.00
4.2.4.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.4.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:46:47
guest_loopback_app testpmd testpmd
4.2.4.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 52014
%usr 0.33
%system 4.30
%guest 0.00
%CPU 4.63
CPU
minflt/s 5.78
majflt/s 0.00
VSZ 6610056
RSS 42081
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 52011
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 52389
%usr 0.00
%system 0.08
%guest 200.07
%CPU 100.16
CPU
minflt/s 1.68
majflt/s 0.00
VSZ 2629997
RSS 220554
%MEM 0.34
kB_rd/s 0.00
kB_wr/s 5.44
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 52365
%usr 0.00
%system 0.03
%guest 200.07
%CPU 100.16
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2657816
RSS 269403
%MEM 0.41
kB_rd/s 0.00
kB_wr/s 0.02
kB_ccwr/s 0.00
4.2.4.7. Anomalies

No anomalies were detected during the course of this test.

4.2.4.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.5. Test ID: PHY2PHY_TPUT

4.2.5.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 513153802b329c0584d72655460a4a264ad41e7c

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

Below are test details:

  • Test ID: phy2phy_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: p2p
  • Traffic type: rfc2544
  • Bidirectional : True
4.2.5.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.5.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 758000.519
tx_rate_mbps Unknown
throughput_rx_mbps 388.096
tx_rate_percent 2.547
throughput_rx_percent 2.547
frame_loss_percent 0.000
min_latency_ns 4020.000
max_latency_ns 6389160.000
avg_latency_ns 461084.000
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:33:06
4.2.5.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 17886
%usr 0.35
%system 4.40
%guest 0.00
%CPU 4.75
CPU
minflt/s 4.88
majflt/s 0.00
VSZ 6609932
RSS 31441
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 17883
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45916
RSS 3232
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.5.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.5.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 691513.887
tx_rate_mbps Unknown
throughput_rx_mbps 708.110
tx_rate_percent 4.094
throughput_rx_percent 4.094
frame_loss_percent 0.000
min_latency_ns 4320.000
max_latency_ns 2901840.000
avg_latency_ns 189268.000
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:33:06
4.2.5.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 17886
%usr 0.35
%system 4.40
%guest 0.00
%CPU 4.75
CPU
minflt/s 4.88
majflt/s 0.00
VSZ 6609932
RSS 31441
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 17883
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45916
RSS 3232
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.5.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.5.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 664871.503
tx_rate_mbps Unknown
throughput_rx_mbps 2723.314
tx_rate_percent 14.148
throughput_rx_percent 14.148
frame_loss_percent 0.000
min_latency_ns 5100.000
max_latency_ns 4001220.000
avg_latency_ns 281681.000
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:33:06
4.2.5.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 17886
%usr 0.35
%system 4.40
%guest 0.00
%CPU 4.75
CPU
minflt/s 4.88
majflt/s 0.00
VSZ 6609932
RSS 31441
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 17883
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45916
RSS 3232
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.5.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.5.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 616639.482
tx_rate_mbps Unknown
throughput_rx_mbps 5051.511
tx_rate_percent 25.750
throughput_rx_percent 25.750
frame_loss_percent 0.000
min_latency_ns 5800.000
max_latency_ns 2885960.000
avg_latency_ns 145821.500
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:33:06
4.2.5.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 17886
%usr 0.35
%system 4.40
%guest 0.00
%CPU 4.75
CPU
minflt/s 4.88
majflt/s 0.00
VSZ 6609932
RSS 31441
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 17883
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45916
RSS 3232
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.5.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.5.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 594574.462
tx_rate_mbps Unknown
throughput_rx_mbps 7220.512
tx_rate_percent 36.578
throughput_rx_percent 36.578
frame_loss_percent 0.000
min_latency_ns 6620.000
max_latency_ns 2268400.000
avg_latency_ns 91636.500
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:33:06
4.2.5.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 17886
%usr 0.35
%system 4.40
%guest 0.00
%CPU 4.75
CPU
minflt/s 4.88
majflt/s 0.00
VSZ 6609932
RSS 31441
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 17883
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45916
RSS 3232
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.5.7. Anomalies

No anomalies were detected during the course of this test.

4.2.5.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.6. Test ID: PHY2PHY_SCALABILITY

4.2.6.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 513153802b329c0584d72655460a4a264ad41e7c

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

Below are test details:

  • Test ID: phy2phy_scalability
  • Description: LTD.Scalability.Flows.RFC2544.0PacketLoss
  • Deployment: p2p
  • Traffic type: rfc2544
  • Bidirectional : True
4.2.6.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.6.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 757882.012
tx_rate_mbps Unknown
throughput_rx_mbps 388.036
tx_rate_percent 2.547
throughput_rx_percent 2.547
frame_loss_percent 0.000
min_latency_ns 4020.000
max_latency_ns 6985500.000
avg_latency_ns 595378.500
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:35:42
stream_count 8000
match_type L4
pre-installed_flows No
4.2.6.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 32072
%usr 0.35
%system 4.42
%guest 0.00
%CPU 4.77
CPU
minflt/s 7.29
majflt/s 0.00
VSZ 6609936
RSS 40207
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 32069
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.6.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.6.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 691513.905
tx_rate_mbps Unknown
throughput_rx_mbps 708.110
tx_rate_percent 4.094
throughput_rx_percent 4.094
frame_loss_percent 0.000
min_latency_ns 4300.000
max_latency_ns 4060680.000
avg_latency_ns 227865.500
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:35:42
stream_count 8000
match_type L4
pre-installed_flows No
4.2.6.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 32072
%usr 0.35
%system 4.42
%guest 0.00
%CPU 4.77
CPU
minflt/s 7.29
majflt/s 0.00
VSZ 6609936
RSS 40207
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 32069
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.6.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.6.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 628539.184
tx_rate_mbps Unknown
throughput_rx_mbps 2574.496
tx_rate_percent 13.375
throughput_rx_percent 13.375
frame_loss_percent 0.000
min_latency_ns 5240.000
max_latency_ns 3140440.000
avg_latency_ns 221741.000
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:35:42
stream_count 8000
match_type L4
pre-installed_flows No
4.2.6.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 32072
%usr 0.35
%system 4.42
%guest 0.00
%CPU 4.77
CPU
minflt/s 7.29
majflt/s 0.00
VSZ 6609936
RSS 40207
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 32069
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.6.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.6.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 579578.155
tx_rate_mbps Unknown
throughput_rx_mbps 4747.904
tx_rate_percent 24.203
throughput_rx_percent 24.203
frame_loss_percent 0.000
min_latency_ns 5680.000
max_latency_ns 1916820.000
avg_latency_ns 100157.000
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:35:42
stream_count 8000
match_type L4
pre-installed_flows No
4.2.6.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 32072
%usr 0.35
%system 4.42
%guest 0.00
%CPU 4.77
CPU
minflt/s 7.29
majflt/s 0.00
VSZ 6609936
RSS 40207
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 32069
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.6.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.6.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 594573.920
tx_rate_mbps Unknown
throughput_rx_mbps 7220.506
tx_rate_percent 36.578
throughput_rx_percent 36.578
frame_loss_percent 0.000
min_latency_ns 6640.000
max_latency_ns 2589480.000
avg_latency_ns 119611.000
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:35:42
stream_count 8000
match_type L4
pre-installed_flows No
4.2.6.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 32072
%usr 0.35
%system 4.42
%guest 0.00
%CPU 4.77
CPU
minflt/s 7.29
majflt/s 0.00
VSZ 6609936
RSS 40207
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 32069
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3259
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.6.7. Anomalies

No anomalies were detected during the course of this test.

4.2.6.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.7. Test ID: PVP_BACK2BACK

4.2.7.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: 513153802b329c0584d72655460a4a264ad41e7c

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • VNF: QemuVirtioNet, Version: 2.5.0, GIT tag: a8c40fa2d667e585382080db36ac44e216b37a1c

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: pvp_back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: pvp
  • Traffic type: rfc2544
  • Bidirectional : True
4.2.7.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.7.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 425
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:46:27
guest_loopback_app testpmd
4.2.7.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 41167
%usr 0.35
%system 4.31
%guest 0.00
%CPU 4.65
CPU
minflt/s 6.10
majflt/s 0.00
VSZ 6609932
RSS 33990
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 41136
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3260
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 41456
%usr 0.00
%system 0.19
%guest 200.34
%CPU 100.16
CPU
minflt/s 1.71
majflt/s 0.00
VSZ 2696728
RSS 286792
%MEM 0.44
kB_rd/s 0.00
kB_wr/s 5.35
kB_ccwr/s 0.00
4.2.7.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.7.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:46:27
guest_loopback_app testpmd
4.2.7.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 41167
%usr 0.35
%system 4.31
%guest 0.00
%CPU 4.65
CPU
minflt/s 6.10
majflt/s 0.00
VSZ 6609932
RSS 33990
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 41136
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3260
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 41456
%usr 0.00
%system 0.19
%guest 200.34
%CPU 100.16
CPU
minflt/s 1.71
majflt/s 0.00
VSZ 2696728
RSS 286792
%MEM 0.44
kB_rd/s 0.00
kB_wr/s 5.35
kB_ccwr/s 0.00
4.2.7.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.7.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1075
b2b_frame_loss_percent 0.0
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:46:27
guest_loopback_app testpmd
4.2.7.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 41167
%usr 0.35
%system 4.31
%guest 0.00
%CPU 4.65
CPU
minflt/s 6.10
majflt/s 0.00
VSZ 6609932
RSS 33990
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 41136
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3260
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 41456
%usr 0.00
%system 0.19
%guest 200.34
%CPU 100.16
CPU
minflt/s 1.71
majflt/s 0.00
VSZ 2696728
RSS 286792
%MEM 0.44
kB_rd/s 0.00
kB_wr/s 5.35
kB_ccwr/s 0.00
4.2.7.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.7.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:46:27
guest_loopback_app testpmd
4.2.7.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 41167
%usr 0.35
%system 4.31
%guest 0.00
%CPU 4.65
CPU
minflt/s 6.10
majflt/s 0.00
VSZ 6609932
RSS 33990
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 41136
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3260
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 41456
%usr 0.00
%system 0.19
%guest 200.34
%CPU 100.16
CPU
minflt/s 1.71
majflt/s 0.00
VSZ 2696728
RSS 286792
%MEM 0.44
kB_rd/s 0.00
kB_wr/s 5.35
kB_ccwr/s 0.00
4.2.7.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.7.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:46:27
guest_loopback_app testpmd
4.2.7.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 41167
%usr 0.35
%system 4.31
%guest 0.00
%CPU 4.65
CPU
minflt/s 6.10
majflt/s 0.00
VSZ 6609932
RSS 33990
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 41136
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45912
RSS 3260
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 41456
%usr 0.00
%system 0.19
%guest 200.34
%CPU 100.16
CPU
minflt/s 1.71
majflt/s 0.00
VSZ 2696728
RSS 286792
%MEM 0.44
kB_rd/s 0.00
kB_wr/s 5.35
kB_ccwr/s 0.00
4.2.7.7. Anomalies

No anomalies were detected during the course of this test.

4.2.7.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.8. Test ID: PVVP_TPUT

4.2.8.1. Test Environment

Below is the environment that the test was performed in:

  • OS: centos 7.2.1511 Core

  • Kernel Version: 3.10.0-327.28.2.el7.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 88

  • Memory: 65687480 kB

  • Virtual Switch Set-up: pvvp

  • vswitchperf: GIT tag: 513153802b329c0584d72655460a4a264ad41e7c

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.6.90, GIT tag: ed26e3ea9995ba632e681d5990af5ee9814f650e

  • VNF: QemuVirtioNet, Version: 2.5.0, GIT tag: a8c40fa2d667e585382080db36ac44e216b37a1c

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325
    • testpmd, Version: 16.07.0, GIT tag: 20e2b6eba13d9eb61b23ea75f09f2aa966fa6325

Below are test details:

  • Test ID: pvvp_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvvp
  • Traffic type: rfc2544
  • Bidirectional : True
4.2.8.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.8.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 523935.894
tx_rate_mbps Unknown
throughput_rx_mbps 268.255
tx_rate_percent 1.773
throughput_rx_percent 1.760
frame_loss_percent 0.737
min_latency_ns 26400.000
max_latency_ns 28224320.000
avg_latency_ns 136804.500
type rfc2544
packet_size 64
traffic_type udp
test_execution_time 00:36:32
guest_loopback_app testpmd testpmd
4.2.8.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 46933
%usr 0.33
%system 4.25
%guest 0.00
%CPU 4.58
CPU
minflt/s 6.10
majflt/s 0.00
VSZ 6610056
RSS 31557
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 46930
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 47447
%usr 0.00
%system 0.83
%guest 199.83
%CPU 100.17
CPU
minflt/s 1.62
majflt/s 0.00
VSZ 2677268
RSS 166273
%MEM 0.25
kB_rd/s 0.00
kB_wr/s 6.98
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 47367
%usr 0.00
%system 0.76
%guest 199.70
%CPU 100.16
CPU
minflt/s 1.17
majflt/s 0.00
VSZ 2598424
RSS 184710
%MEM 0.28
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
4.2.8.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.8.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 299562.766
tx_rate_mbps Unknown
throughput_rx_mbps 306.752
tx_rate_percent 1.773
throughput_rx_percent 1.773
frame_loss_percent 0.002
min_latency_ns 21900.000
max_latency_ns 1851260.000
avg_latency_ns 55877.500
type rfc2544
packet_size 128
traffic_type udp
test_execution_time 00:36:32
guest_loopback_app testpmd testpmd
4.2.8.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 46933
%usr 0.33
%system 4.25
%guest 0.00
%CPU 4.58
CPU
minflt/s 6.10
majflt/s 0.00
VSZ 6610056
RSS 31557
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 46930
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 47447
%usr 0.00
%system 0.83
%guest 199.83
%CPU 100.17
CPU
minflt/s 1.62
majflt/s 0.00
VSZ 2677268
RSS 166273
%MEM 0.25
kB_rd/s 0.00
kB_wr/s 6.98
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 47367
%usr 0.00
%system 0.76
%guest 199.70
%CPU 100.16
CPU
minflt/s 1.17
majflt/s 0.00
VSZ 2598424
RSS 184710
%MEM 0.28
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
4.2.8.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.8.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 228722.683
tx_rate_mbps Unknown
throughput_rx_mbps 936.848
tx_rate_percent 4.867
throughput_rx_percent 4.867
frame_loss_percent 0.000
min_latency_ns 45360.000
max_latency_ns 1077480.000
avg_latency_ns 123971.500
type rfc2544
packet_size 512
traffic_type udp
test_execution_time 00:36:32
guest_loopback_app testpmd testpmd
4.2.8.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 46933
%usr 0.33
%system 4.25
%guest 0.00
%CPU 4.58
CPU
minflt/s 6.10
majflt/s 0.00
VSZ 6610056
RSS 31557
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 46930
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 47447
%usr 0.00
%system 0.83
%guest 199.83
%CPU 100.17
CPU
minflt/s 1.62
majflt/s 0.00
VSZ 2677268
RSS 166273
%MEM 0.25
kB_rd/s 0.00
kB_wr/s 6.98
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 47367
%usr 0.00
%system 0.76
%guest 199.70
%CPU 100.16
CPU
minflt/s 1.17
majflt/s 0.00
VSZ 2598424
RSS 184710
%MEM 0.28
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
4.2.8.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.8.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 246199.575
tx_rate_mbps Unknown
throughput_rx_mbps 2016.867
tx_rate_percent 10.281
throughput_rx_percent 10.281
frame_loss_percent 0.000
min_latency_ns 50040.000
max_latency_ns 723740.000
avg_latency_ns 129189.000
type rfc2544
packet_size 1024
traffic_type udp
test_execution_time 00:36:32
guest_loopback_app testpmd testpmd
4.2.8.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 46933
%usr 0.33
%system 4.25
%guest 0.00
%CPU 4.58
CPU
minflt/s 6.10
majflt/s 0.00
VSZ 6610056
RSS 31557
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 46930
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 47447
%usr 0.00
%system 0.83
%guest 199.83
%CPU 100.17
CPU
minflt/s 1.62
majflt/s 0.00
VSZ 2677268
RSS 166273
%MEM 0.25
kB_rd/s 0.00
kB_wr/s 6.98
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 47367
%usr 0.00
%system 0.76
%guest 199.70
%CPU 100.16
CPU
minflt/s 1.17
majflt/s 0.00
VSZ 2598424
RSS 184710
%MEM 0.28
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
4.2.8.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.8.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 267698.581
tx_rate_mbps Unknown
throughput_rx_mbps 3250.932
tx_rate_percent 16.469
throughput_rx_percent 16.469
frame_loss_percent 0.000
min_latency_ns 59520.000
max_latency_ns 2039500.000
avg_latency_ns 134039.000
type rfc2544
packet_size 1518
traffic_type udp
test_execution_time 00:36:32
guest_loopback_app testpmd testpmd
4.2.8.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 46933
%usr 0.33
%system 4.25
%guest 0.00
%CPU 4.58
CPU
minflt/s 6.10
majflt/s 0.00
VSZ 6610056
RSS 31557
%MEM 0.05
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 46930
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 45908
RSS 3268
%MEM 0.00
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 47447
%usr 0.00
%system 0.83
%guest 199.83
%CPU 100.17
CPU
minflt/s 1.62
majflt/s 0.00
VSZ 2677268
RSS 166273
%MEM 0.25
kB_rd/s 0.00
kB_wr/s 6.98
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 47367
%usr 0.00
%system 0.76
%guest 199.70
%CPU 100.16
CPU
minflt/s 1.17
majflt/s 0.00
VSZ 2598424
RSS 184710
%MEM 0.28
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
4.2.8.7. Anomalies

No anomalies were detected during the course of this test.

4.2.8.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.3. Rationale for decisions

The tests conducted do not have pass/fail/conditional-pass criteria. The test is simply conducted and the results are reported.

4.4. Conclusions and recommendations

The test results are stable. The vsperf CI jobs that were used to obtain the results can be found at https://artifacts.opnfv.org/logs/vswitchperf/intel-pod3/2016-12-02_10-35-07/vswitchperf_logs_2016-12-02_10-35-07.tar.gz.

4.5. General

4.5.1. Glossary

  • NFV - Network Function Virtualization
  • Mbps - 1,000,000bps

4.5.2. Document change procedures and history

Document ID Author Date Modified
OPNFV_vswitchperf_LTR_ver_1.0_Jan_15_CN_DRAFT Christopher Nolan 23/01/2015
OPNFV_vswitchperf_LTR_ver_1.1_Jan_15_CN_DRAFT Christopher Nolan 28/01/2015