VSPERF Results

1. VSPERF Test Scenarios

Predefined Tests suitable for automated execution with CI:

Test Definition
phy2phy_tput PacketLossRatio for Phy2Phy
back2back BackToBackFrames for Phy2Phy
phy2phy_tput_mod_vlan PacketLossRatioFrameModification for Phy2Phy
phy2phy_cont Phy2Phy blast vswitch at x% TX rate and measure throughput
pvp_cont PVP blast vswitch at x% TX rate and measure throughput
pvvp_cont PVVP blast vswitch at x% TX rate and measure throughput
phy2phy_scalability Scalability0PacketLoss for Phy2Phy
pvp_tput PacketLossRatio for PVP
pvp_back2back BackToBackFrames for PVP
pvvp_tput PacketLossRatio for PVVP
pvvp_back2back BackToBackFrames for PVVP
phy2phy_cpu_load CPU0PacketLoss for Phy2Phy
phy2phy_mem_load Same as CPU0PacketLoss but using a memory intensive app

Deployment topologies:

  • Phy2Phy: Physical port -> vSwitch -> Physical port.
  • PVP: Physical port -> vSwitch -> VNF -> vSwitch -> Physical port.
  • PVVP: Physical port -> vSwitch -> VNF -> vSwitch -> VNF -> vSwitch -> Physical port.

Loopback applications in the Guest:

Supported traffic generators:

  • Spirent Testcenter
  • Ixia: IxOS and IxNet.
  • Xena
  • MoonGen
  • Dummy
  • T-Rex

2. OPNFV Test Results

VSPERF CI jobs are run daily and sample results can be found at https://wiki.opnfv.org/display/vsperf/Vsperf+Results

Testcase names shown in the dashboard are combination of orignal testcase name from VSPERF framework and indication of used vswitch.

Example:

Testcase phy2phy_tput is executed for three vSwitch types: OvsDpdkVhost, OvsVanilla and VppDpdkVhost. In this case, following testcase names will be used in the dashboard: phy2phy_tput_ovsdpdkvhost, phy2phy_tput_ovsvanilla and phy2phy_tput_vppdpdkvhost.

In case of RFC2544 Throughput test, the recorded metric is FPS (frames per second) without packet loss. For RFC2544 Back2Back test, the recorded metric is back-to-back value (number of frames) without packet loss.

The loopback application in the VNF used for PVP and PVVP scenarios was DPDK testpmd.

Guest interface types are vhost-user for OvsDpdkVhost and VppDpdkVhost and virtio-net for OvsVanilla.

3. Performance report for OvsDpdkVhost

3.1. Introduction

The objective of the OPNFV project titled “Characterise vSwitch Performance for Telco NFV Use Cases”, is to evaluate a virtual switch to identify its suitability for a Telco Network Function Virtualization (NFV) environment. As well as this, the project aims to identify any gaps or bottlenecks in order to drive architectural changes to improve virtual switch performance and determinism. The purpose of this document is to summarize the results of the tests carried out on the virtual switch in the Network Function Virtualization Infrastructure (NFVI) and, from these results, provide evaluations and recommendations for the virtual switch. Test results will be outlined in details-of-LTR, preceded by the document-identifier and the scope and references).

This document is currently in draft form.

3.1.1. Document identifier

The document id will be used to uniquely identify versions of the LTR. The format for the document id will be: OPNFV_vswitchperf_LTR_rel_STATUS, the status is one of: DRAFT, REVIEWED, CORRECTED or FINAL. The document id for this version of the LTR is: OPNFV_vswitchperf_LTR_Brahmaputra_DRAFT.

3.1.2. Scope

The scope of this report is to detail the results of the tests that have been performed on the virtual switch. This report will also evaluate the results of these tests and, based on these evaluations, provide recommendations on the suitability of the virtual switch for use in a Telco NFV environment.

3.2. Details of the Level Test Report

This section provides a test-results-overview. Also included are the rationale and the conclusions.

3.2.1. Test ID: BACK2BACK

3.2.1.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 3daf5942e2ede8088898a9589d41b797e27b6d4e

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: p2p
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
3.2.1.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.1.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 20861
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 64
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:37:49
start_time 2018-06-04 20:26:00
stop_time 2018-06-04 21:03:49
3.2.1.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.15
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056812
RSS 42603
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50704
RSS 6736
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.1.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 253378378
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 128
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:37:49
start_time 2018-06-04 20:26:00
stop_time 2018-06-04 21:03:49
3.2.1.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.15
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056812
RSS 42603
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50704
RSS 6736
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.1.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 70488721
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:37:49
start_time 2018-06-04 20:26:00
stop_time 2018-06-04 21:03:49
3.2.1.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.15
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056812
RSS 42603
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50704
RSS 6736
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.1.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 35919540
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:37:49
start_time 2018-06-04 20:26:00
stop_time 2018-06-04 21:03:49
3.2.1.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.15
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056812
RSS 42603
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50704
RSS 6736
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.1.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 24382314
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:37:49
start_time 2018-06-04 20:26:00
stop_time 2018-06-04 21:03:49
3.2.1.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.15
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056812
RSS 42603
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50704
RSS 6736
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.7. Anomalies

No anomalies were detected during the course of this test.

3.2.1.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.2. Test ID: PHY2PHY_SCALABILITY

3.2.2.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 3daf5942e2ede8088898a9589d41b797e27b6d4e

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: phy2phy_scalability
  • Description: LTD.Scalability.Flows.RFC2544.0PacketLoss
  • Deployment: p2p
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
3.2.2.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.2.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 15490176.197
tx_rate_mbps Unknown
throughput_rx_mbps 7930.970
tx_rate_percent 52.047
throughput_rx_percent 52.047
frame_loss_percent 0.000
min_latency_ns 5520.000
max_latency_ns 89060.000
avg_latency_ns 13522.000
type rfc2544_throughput
packet_size 64
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:22:26
start_time 2018-06-04 21:34:05
stop_time 2018-06-04 21:56:31
stream_count 8000
match_type L4
pre-installed_flows No
3.2.2.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.15
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 5056816
RSS 42990
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6660
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.2.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.2.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 15585423.269
tx_rate_mbps Unknown
throughput_rx_mbps 15959.473
tx_rate_percent 92.266
throughput_rx_percent 92.266
frame_loss_percent 0.000
min_latency_ns 5440.000
max_latency_ns 84580.000
avg_latency_ns 16248.000
type rfc2544_throughput
packet_size 128
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:22:26
start_time 2018-06-04 21:34:05
stop_time 2018-06-04 21:56:31
stream_count 8000
match_type L4
pre-installed_flows No
3.2.2.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.15
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 5056816
RSS 42990
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6660
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.2.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.2.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699198.853
tx_rate_mbps Unknown
throughput_rx_mbps 19247.919
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 15820.000
max_latency_ns 50340.000
avg_latency_ns 37476.000
type rfc2544_throughput
packet_size 512
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:22:26
start_time 2018-06-04 21:34:05
stop_time 2018-06-04 21:56:31
stream_count 8000
match_type L4
pre-installed_flows No
3.2.2.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.15
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 5056816
RSS 42990
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6660
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.2.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.2.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394614.492
tx_rate_mbps Unknown
throughput_rx_mbps 19616.682
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 18500.000
max_latency_ns 49060.000
avg_latency_ns 35227.000
type rfc2544_throughput
packet_size 1024
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:22:26
start_time 2018-06-04 21:34:05
stop_time 2018-06-04 21:56:31
stream_count 8000
match_type L4
pre-installed_flows No
3.2.2.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.15
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 5056816
RSS 42990
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6660
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.2.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.2.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625474.194
tx_rate_mbps Unknown
throughput_rx_mbps 19739.759
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 17200.000
max_latency_ns 44940.000
avg_latency_ns 32005.500
type rfc2544_throughput
packet_size 1518
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:22:26
start_time 2018-06-04 21:34:05
stop_time 2018-06-04 21:56:31
stream_count 8000
match_type L4
pre-installed_flows No
3.2.2.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.15
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 5056816
RSS 42990
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6660
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.2.7. Anomalies

No anomalies were detected during the course of this test.

3.2.2.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.3. Test ID: PHY2PHY_TPUT_MOD_VLAN

3.2.3.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 3daf5942e2ede8088898a9589d41b797e27b6d4e

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: phy2phy_tput_mod_vlan
  • Description: LTD.Throughput.RFC2544.PacketLossRatioFrameModification
  • Deployment: p2p
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
3.2.3.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.3.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 14569458.593
tx_rate_mbps Unknown
throughput_rx_mbps 7925.785
tx_rate_percent 48.953
throughput_rx_percent 51.284
frame_loss_percent 0.000
min_latency_ns 5180.000
max_latency_ns 47720.000
avg_latency_ns 12268.500
type rfc2544_throughput
packet_size 64
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:30:15
start_time 2018-06-04 21:03:49
stop_time 2018-06-04 21:34:05
3.2.3.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.15
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056816
RSS 43071
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6660
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.3.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.3.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 14670903.494
tx_rate_mbps Unknown
throughput_rx_mbps 15492.474
tx_rate_percent 86.852
throughput_rx_percent 89.199
frame_loss_percent 0.000
min_latency_ns 5500.000
max_latency_ns 44320.000
avg_latency_ns 15569.500
type rfc2544_throughput
packet_size 128
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:30:15
start_time 2018-06-04 21:03:49
stop_time 2018-06-04 21:34:05
3.2.3.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.15
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056816
RSS 43071
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6660
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.3.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.3.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4662904.152
tx_rate_mbps Unknown
throughput_rx_mbps 19248.468
tx_rate_percent 99.227
throughput_rx_percent 99.973
frame_loss_percent 0.000
min_latency_ns 6200.000
max_latency_ns 23580.000
avg_latency_ns 10882.000
type rfc2544_throughput
packet_size 512
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:30:15
start_time 2018-06-04 21:03:49
stop_time 2018-06-04 21:34:05
3.2.3.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.15
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056816
RSS 43071
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6660
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.3.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.3.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2376116.893
tx_rate_mbps Unknown
throughput_rx_mbps 19541.185
tx_rate_percent 99.227
throughput_rx_percent 99.607
frame_loss_percent 0.000
min_latency_ns 6620.000
max_latency_ns 23080.000
avg_latency_ns 9346.000
type rfc2544_throughput
packet_size 1024
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:30:15
start_time 2018-06-04 21:03:49
stop_time 2018-06-04 21:34:05
3.2.3.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.15
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056816
RSS 43071
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6660
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.3.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.3.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1612916.327
tx_rate_mbps Unknown
throughput_rx_mbps 19638.869
tx_rate_percent 99.227
throughput_rx_percent 99.485
frame_loss_percent 0.000
min_latency_ns 6880.000
max_latency_ns 23600.000
avg_latency_ns 9509.000
type rfc2544_throughput
packet_size 1518
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:30:15
start_time 2018-06-04 21:03:49
stop_time 2018-06-04 21:34:05
3.2.3.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.15
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056816
RSS 43071
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6660
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.3.7. Anomalies

No anomalies were detected during the course of this test.

3.2.3.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.4. Test ID: PVP_BACK2BACK

3.2.4.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: 3daf5942e2ede8088898a9589d41b797e27b6d4e

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

  • VNF: QemuDpdkVhostUser, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-16.04_trex_20180209.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvp_back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: pvp
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
3.2.4.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.4.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 2128
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 64
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:33:52
start_time 2018-06-04 22:21:52
stop_time 2018-06-04 22:55:44
guest_loopback_app testpmd
3.2.4.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.15
%system 0.02
%guest 0.00
%CPU 400.18
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 11422004
RSS 43020
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6672
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.11
%system 0.11
%guest 200.06
%CPU 200.27
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2812958
RSS 68883
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 11.25
kB_ccwr/s 0.00
3.2.4.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.4.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1449
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 128
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:33:52
start_time 2018-06-04 22:21:52
stop_time 2018-06-04 22:55:44
guest_loopback_app testpmd
3.2.4.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.15
%system 0.02
%guest 0.00
%CPU 400.18
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 11422004
RSS 43020
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6672
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.11
%system 0.11
%guest 200.06
%CPU 200.27
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2812958
RSS 68883
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 11.25
kB_ccwr/s 0.00
3.2.4.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.4.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 70488721
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:33:52
start_time 2018-06-04 22:21:52
stop_time 2018-06-04 22:55:44
guest_loopback_app testpmd
3.2.4.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.15
%system 0.02
%guest 0.00
%CPU 400.18
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 11422004
RSS 43020
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6672
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.11
%system 0.11
%guest 200.06
%CPU 200.27
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2812958
RSS 68883
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 11.25
kB_ccwr/s 0.00
3.2.4.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.4.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 35919540
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:33:52
start_time 2018-06-04 22:21:52
stop_time 2018-06-04 22:55:44
guest_loopback_app testpmd
3.2.4.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.15
%system 0.02
%guest 0.00
%CPU 400.18
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 11422004
RSS 43020
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6672
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.11
%system 0.11
%guest 200.06
%CPU 200.27
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2812958
RSS 68883
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 11.25
kB_ccwr/s 0.00
3.2.4.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.4.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 24382314
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:33:52
start_time 2018-06-04 22:21:52
stop_time 2018-06-04 22:55:44
guest_loopback_app testpmd
3.2.4.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.15
%system 0.02
%guest 0.00
%CPU 400.18
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 11422004
RSS 43020
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6672
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.11
%system 0.11
%guest 200.06
%CPU 200.27
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2812958
RSS 68883
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 11.25
kB_ccwr/s 0.00
3.2.4.7. Anomalies

No anomalies were detected during the course of this test.

3.2.4.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.5. Test ID: PHY2PHY_TPUT

3.2.5.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 3daf5942e2ede8088898a9589d41b797e27b6d4e

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: phy2phy_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: p2p
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
3.2.5.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.5.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 19633555.001
tx_rate_mbps Unknown
throughput_rx_mbps 10052.380
tx_rate_percent 65.969
throughput_rx_percent 65.969
frame_loss_percent 0.000
min_latency_ns 5100.000
max_latency_ns 38560.000
avg_latency_ns 10737.500
type rfc2544_throughput
packet_size 64
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:20:28
start_time 2018-06-04 20:05:30
stop_time 2018-06-04 20:25:59
3.2.5.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.12
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 5056816
RSS 43054
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50709
RSS 6799
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.5.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.5.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 16891641.844
tx_rate_mbps Unknown
throughput_rx_mbps 17297.041
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 16440.000
max_latency_ns 72940.000
avg_latency_ns 50757.000
type rfc2544_throughput
packet_size 128
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:20:28
start_time 2018-06-04 20:05:30
stop_time 2018-06-04 20:25:59
3.2.5.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.12
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 5056816
RSS 43054
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50709
RSS 6799
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.5.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.5.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699202.096
tx_rate_mbps Unknown
throughput_rx_mbps 19247.932
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 16880.000
max_latency_ns 46840.000
avg_latency_ns 36184.500
type rfc2544_throughput
packet_size 512
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:20:28
start_time 2018-06-04 20:05:30
stop_time 2018-06-04 20:25:59
3.2.5.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.12
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 5056816
RSS 43054
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50709
RSS 6799
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.5.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.5.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394615.131
tx_rate_mbps Unknown
throughput_rx_mbps 19616.687
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 15780.000
max_latency_ns 43120.000
avg_latency_ns 32234.500
type rfc2544_throughput
packet_size 1024
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:20:28
start_time 2018-06-04 20:05:30
stop_time 2018-06-04 20:25:59
3.2.5.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.12
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 5056816
RSS 43054
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50709
RSS 6799
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.5.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.5.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625474.492
tx_rate_mbps Unknown
throughput_rx_mbps 19739.762
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 15580.000
max_latency_ns 41340.000
avg_latency_ns 30292.000
type rfc2544_throughput
packet_size 1518
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:20:28
start_time 2018-06-04 20:05:30
stop_time 2018-06-04 20:25:59
3.2.5.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.12
%system 0.02
%guest 0.00
%CPU 200.18
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 5056816
RSS 43054
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50709
RSS 6799
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.5.7. Anomalies

No anomalies were detected during the course of this test.

3.2.5.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.6. Test ID: PVP_TPUT

3.2.6.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: 3daf5942e2ede8088898a9589d41b797e27b6d4e

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

  • VNF: QemuDpdkVhostUser, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-16.04_trex_20180209.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvp_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvp
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
3.2.6.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.6.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 3980658.127
tx_rate_mbps Unknown
throughput_rx_mbps 2038.097
tx_rate_percent 13.375
throughput_rx_percent 13.375
frame_loss_percent 0.000
min_latency_ns 7080.000
max_latency_ns 43440.000
avg_latency_ns 10295.500
type rfc2544_throughput
packet_size 64
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:25:19
start_time 2018-06-04 21:56:32
stop_time 2018-06-04 22:21:51
guest_loopback_app testpmd
3.2.6.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.15
%system 0.02
%guest 0.00
%CPU 400.18
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 11422004
RSS 43084
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6248
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.08
%system 0.07
%guest 200.04
%CPU 200.20
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 2834575
RSS 65376
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 15.15
kB_ccwr/s 0.00
3.2.6.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.6.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 8530430.372
tx_rate_mbps Unknown
throughput_rx_mbps 8735.161
tx_rate_percent 50.500
throughput_rx_percent 50.500
frame_loss_percent 0.000
min_latency_ns 13220.000
max_latency_ns 44020.000
avg_latency_ns 17266.500
type rfc2544_throughput
packet_size 128
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:25:19
start_time 2018-06-04 21:56:32
stop_time 2018-06-04 22:21:51
guest_loopback_app testpmd
3.2.6.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.15
%system 0.02
%guest 0.00
%CPU 400.18
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 11422004
RSS 43084
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6248
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.08
%system 0.07
%guest 200.04
%CPU 200.20
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 2834575
RSS 65376
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 15.15
kB_ccwr/s 0.00
3.2.6.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.6.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699177.190
tx_rate_mbps Unknown
throughput_rx_mbps 19247.830
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 27020.000
max_latency_ns 73020.000
avg_latency_ns 62402.000
type rfc2544_throughput
packet_size 512
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:25:19
start_time 2018-06-04 21:56:32
stop_time 2018-06-04 22:21:51
guest_loopback_app testpmd
3.2.6.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.15
%system 0.02
%guest 0.00
%CPU 400.18
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 11422004
RSS 43084
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6248
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.08
%system 0.07
%guest 200.04
%CPU 200.20
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 2834575
RSS 65376
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 15.15
kB_ccwr/s 0.00
3.2.6.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.6.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394607.708
tx_rate_mbps Unknown
throughput_rx_mbps 19616.626
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 29120.000
max_latency_ns 67840.000
avg_latency_ns 54068.000
type rfc2544_throughput
packet_size 1024
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:25:19
start_time 2018-06-04 21:56:32
stop_time 2018-06-04 22:21:51
guest_loopback_app testpmd
3.2.6.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.15
%system 0.02
%guest 0.00
%CPU 400.18
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 11422004
RSS 43084
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6248
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.08
%system 0.07
%guest 200.04
%CPU 200.20
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 2834575
RSS 65376
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 15.15
kB_ccwr/s 0.00
3.2.6.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.6.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625468.971
tx_rate_mbps Unknown
throughput_rx_mbps 19739.695
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 29240.000
max_latency_ns 67380.000
avg_latency_ns 52108.000
type rfc2544_throughput
packet_size 1518
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:25:19
start_time 2018-06-04 21:56:32
stop_time 2018-06-04 22:21:51
guest_loopback_app testpmd
3.2.6.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.15
%system 0.02
%guest 0.00
%CPU 400.18
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 11422004
RSS 43084
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6248
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.08
%system 0.07
%guest 200.04
%CPU 200.20
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 2834575
RSS 65376
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 15.15
kB_ccwr/s 0.00
3.2.6.7. Anomalies

No anomalies were detected during the course of this test.

3.2.6.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.3. Rationale for decisions

The tests conducted do not have pass/fail/conditional-pass criteria. The test is simply conducted and the results are reported.

3.4. Conclusions and recommendations

The test results are stable. The vsperf CI jobs that were used to obtain the results can be found at https://artifacts.opnfv.org/logs/vswitchperf/intel-pod12/2018-06-05_03-05-23/vswitchperf_logs_2018-06-05_03-05-23.tar.gz.

3.5. General

3.5.1. Glossary

  • NFV - Network Function Virtualization
  • Mbps - 1,000,000bps

3.5.2. Document change procedures and history

Document ID Author Date Modified
OPNFV_vswitchperf_LTR_ver_1.0_Jan_15_CN_DRAFT Christopher Nolan 23/01/2015
OPNFV_vswitchperf_LTR_ver_1.1_Jan_15_CN_DRAFT Christopher Nolan 28/01/2015

4. Performance report for OvsVanilla

4.1. Introduction

The objective of the OPNFV project titled “Characterise vSwitch Performance for Telco NFV Use Cases”, is to evaluate a virtual switch to identify its suitability for a Telco Network Function Virtualization (NFV) environment. As well as this, the project aims to identify any gaps or bottlenecks in order to drive architectural changes to improve virtual switch performance and determinism. The purpose of this document is to summarize the results of the tests carried out on the virtual switch in the Network Function Virtualization Infrastructure (NFVI) and, from these results, provide evaluations and recommendations for the virtual switch. Test results will be outlined in details-of-LTR, preceded by the document-identifier and the scope and references).

This document is currently in draft form.

4.1.1. Document identifier

The document id will be used to uniquely identify versions of the LTR. The format for the document id will be: OPNFV_vswitchperf_LTR_rel_STATUS, the status is one of: DRAFT, REVIEWED, CORRECTED or FINAL. The document id for this version of the LTR is: OPNFV_vswitchperf_LTR_Brahmaputra_DRAFT.

4.1.2. Scope

The scope of this report is to detail the results of the tests that have been performed on the virtual switch. This report will also evaluate the results of these tests and, based on these evaluations, provide recommendations on the suitability of the virtual switch for use in a Telco NFV environment.

4.2. Details of the Level Test Report

This section provides a test-results-overview. Also included are the rationale and the conclusions.

4.2.1. Test ID: BACK2BACK

4.2.1.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 3daf5942e2ede8088898a9589d41b797e27b6d4e

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

Below are test details:

  • Test ID: back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: p2p
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
4.2.1.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.1.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 5108
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:42:23
start_time 2018-06-04 23:41:20
stop_time 2018-06-05 00:23:44
4.2.1.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.47
%system 1.06
%guest 0.00
%CPU 1.56
CPU
minflt/s 0.12
majflt/s 0.00
VSZ 3364108
RSS 7801
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5868
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.1.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 3382
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:42:23
start_time 2018-06-04 23:41:20
stop_time 2018-06-05 00:23:44
4.2.1.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.47
%system 1.06
%guest 0.00
%CPU 1.56
CPU
minflt/s 0.12
majflt/s 0.00
VSZ 3364108
RSS 7801
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5868
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.1.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1613
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:42:23
start_time 2018-06-04 23:41:20
stop_time 2018-06-05 00:23:44
4.2.1.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.47
%system 1.06
%guest 0.00
%CPU 1.56
CPU
minflt/s 0.12
majflt/s 0.00
VSZ 3364108
RSS 7801
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5868
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.1.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1370
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:42:23
start_time 2018-06-04 23:41:20
stop_time 2018-06-05 00:23:44
4.2.1.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.47
%system 1.06
%guest 0.00
%CPU 1.56
CPU
minflt/s 0.12
majflt/s 0.00
VSZ 3364108
RSS 7801
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5868
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.1.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1488
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:42:23
start_time 2018-06-04 23:41:20
stop_time 2018-06-05 00:23:44
4.2.1.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.47
%system 1.06
%guest 0.00
%CPU 1.56
CPU
minflt/s 0.12
majflt/s 0.00
VSZ 3364108
RSS 7801
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5868
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.7. Anomalies

No anomalies were detected during the course of this test.

4.2.1.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.2. Test ID: PHY2PHY_SCALABILITY

4.2.2.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 3daf5942e2ede8088898a9589d41b797e27b6d4e

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

Below are test details:

  • Test ID: phy2phy_scalability
  • Description: LTD.Scalability.Flows.RFC2544.0PacketLoss
  • Deployment: p2p
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
4.2.2.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.2.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 757999.238
tx_rate_mbps Unknown
throughput_rx_mbps 388.096
tx_rate_percent 2.547
throughput_rx_percent 2.547
frame_loss_percent 0.000
min_latency_ns 4040.000
max_latency_ns 1113620.000
avg_latency_ns 28743.000
type rfc2544_throughput
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:38
start_time 2018-06-05 00:51:28
stop_time 2018-06-05 01:19:06
stream_count 8000
match_type L4
pre-installed_flows No
4.2.2.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.50
%system 1.14
%guest 0.00
%CPU 1.67
CPU
minflt/s 3.63
majflt/s 0.00
VSZ 3364104
RSS 13435
%MEM 0.02
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5852
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.2.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.2.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 822160.828
tx_rate_mbps Unknown
throughput_rx_mbps 841.893
tx_rate_percent 4.867
throughput_rx_percent 4.867
frame_loss_percent 0.000
min_latency_ns 4200.000
max_latency_ns 248660.000
avg_latency_ns 29846.000
type rfc2544_throughput
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:38
start_time 2018-06-05 00:51:28
stop_time 2018-06-05 01:19:06
stream_count 8000
match_type L4
pre-installed_flows No
4.2.2.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.50
%system 1.14
%guest 0.00
%CPU 1.67
CPU
minflt/s 3.63
majflt/s 0.00
VSZ 3364104
RSS 13435
%MEM 0.02
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5852
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.2.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.2.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 810253.894
tx_rate_mbps Unknown
throughput_rx_mbps 3318.800
tx_rate_percent 17.242
throughput_rx_percent 17.242
frame_loss_percent 0.000
min_latency_ns 5040.000
max_latency_ns 726360.000
avg_latency_ns 29733.000
type rfc2544_throughput
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:38
start_time 2018-06-05 00:51:28
stop_time 2018-06-05 01:19:06
stream_count 8000
match_type L4
pre-installed_flows No
4.2.2.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.50
%system 1.14
%guest 0.00
%CPU 1.67
CPU
minflt/s 3.63
majflt/s 0.00
VSZ 3364104
RSS 13435
%MEM 0.02
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5852
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.2.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.2.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 838870.999
tx_rate_mbps Unknown
throughput_rx_mbps 6872.031
tx_rate_percent 35.031
throughput_rx_percent 35.031
frame_loss_percent 0.000
min_latency_ns 5800.000
max_latency_ns 1575660.000
avg_latency_ns 42592.000
type rfc2544_throughput
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:38
start_time 2018-06-05 00:51:28
stop_time 2018-06-05 01:19:06
stream_count 8000
match_type L4
pre-installed_flows No
4.2.2.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.50
%system 1.14
%guest 0.00
%CPU 1.67
CPU
minflt/s 3.63
majflt/s 0.00
VSZ 3364104
RSS 13435
%MEM 0.02
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5852
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.2.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.2.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 833444.024
tx_rate_mbps Unknown
throughput_rx_mbps 10121.344
tx_rate_percent 51.273
throughput_rx_percent 51.273
frame_loss_percent 0.000
min_latency_ns 7620.000
max_latency_ns 363380.000
avg_latency_ns 37769.000
type rfc2544_throughput
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:38
start_time 2018-06-05 00:51:28
stop_time 2018-06-05 01:19:06
stream_count 8000
match_type L4
pre-installed_flows No
4.2.2.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.50
%system 1.14
%guest 0.00
%CPU 1.67
CPU
minflt/s 3.63
majflt/s 0.00
VSZ 3364104
RSS 13435
%MEM 0.02
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5852
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.2.7. Anomalies

No anomalies were detected during the course of this test.

4.2.2.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.3. Test ID: PVVP_TPUT

4.2.3.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvvp

  • vswitchperf: GIT tag: 3daf5942e2ede8088898a9589d41b797e27b6d4e

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • VNF: QemuVirtioNet, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-16.04_trex_20180209.qcow2
    • /home/jenkins/vloop-vnf-ubuntu-16.04_trex_20180209.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvvp_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvvp
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
4.2.3.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.3.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 242626.889
tx_rate_mbps Unknown
throughput_rx_mbps 124.225
tx_rate_percent 1.773
throughput_rx_percent 0.815
frame_loss_percent 54.029
min_latency_ns 37940.000
max_latency_ns 72563300.000
avg_latency_ns 4881328.500
type rfc2544_throughput
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:31:22
start_time 2018-06-05 02:31:33
stop_time 2018-06-05 03:02:56
guest_loopback_app testpmd testpmd
4.2.3.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.51
%system 0.95
%guest 0.00
%CPU 1.47
CPU
minflt/s 0.17
majflt/s 0.00
VSZ 3364228
RSS 8071
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5808
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.04
%system 1.92
%guest 198.17
%CPU 200.13
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 2783674
RSS 68472
%MEM 0.10
kB_rd/s 0.07
kB_wr/s 12.40
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 1.80
%guest 198.27
%CPU 200.09
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2850364
RSS 70796
%MEM 0.11
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.3.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.3.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 299569.564
tx_rate_mbps Unknown
throughput_rx_mbps 306.759
tx_rate_percent 1.773
throughput_rx_percent 1.773
frame_loss_percent 0.000
min_latency_ns 21840.000
max_latency_ns 1799960.000
avg_latency_ns 73677.500
type rfc2544_throughput
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:31:22
start_time 2018-06-05 02:31:33
stop_time 2018-06-05 03:02:56
guest_loopback_app testpmd testpmd
4.2.3.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.51
%system 0.95
%guest 0.00
%CPU 1.47
CPU
minflt/s 0.17
majflt/s 0.00
VSZ 3364228
RSS 8071
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5808
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.04
%system 1.92
%guest 198.17
%CPU 200.13
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 2783674
RSS 68472
%MEM 0.10
kB_rd/s 0.07
kB_wr/s 12.40
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 1.80
%guest 198.27
%CPU 200.09
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2850364
RSS 70796
%MEM 0.11
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.3.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.3.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 228722.504
tx_rate_mbps Unknown
throughput_rx_mbps 936.847
tx_rate_percent 4.867
throughput_rx_percent 4.867
frame_loss_percent 0.000
min_latency_ns 25320.000
max_latency_ns 626800.000
avg_latency_ns 88493.000
type rfc2544_throughput
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:31:22
start_time 2018-06-05 02:31:33
stop_time 2018-06-05 03:02:56
guest_loopback_app testpmd testpmd
4.2.3.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.51
%system 0.95
%guest 0.00
%CPU 1.47
CPU
minflt/s 0.17
majflt/s 0.00
VSZ 3364228
RSS 8071
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5808
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.04
%system 1.92
%guest 198.17
%CPU 200.13
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 2783674
RSS 68472
%MEM 0.10
kB_rd/s 0.07
kB_wr/s 12.40
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 1.80
%guest 198.27
%CPU 200.09
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2850364
RSS 70796
%MEM 0.11
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.3.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.3.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 283242.318
tx_rate_mbps Unknown
throughput_rx_mbps 2320.321
tx_rate_percent 11.828
throughput_rx_percent 11.828
frame_loss_percent 0.000
min_latency_ns 29740.000
max_latency_ns 943520.000
avg_latency_ns 100687.500
type rfc2544_throughput
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:31:22
start_time 2018-06-05 02:31:33
stop_time 2018-06-05 03:02:56
guest_loopback_app testpmd testpmd
4.2.3.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.51
%system 0.95
%guest 0.00
%CPU 1.47
CPU
minflt/s 0.17
majflt/s 0.00
VSZ 3364228
RSS 8071
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5808
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.04
%system 1.92
%guest 198.17
%CPU 200.13
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 2783674
RSS 68472
%MEM 0.10
kB_rd/s 0.07
kB_wr/s 12.40
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 1.80
%guest 198.27
%CPU 200.09
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2850364
RSS 70796
%MEM 0.11
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.3.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.3.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 280271.581
tx_rate_mbps Unknown
throughput_rx_mbps 3403.618
tx_rate_percent 17.242
throughput_rx_percent 17.242
frame_loss_percent 0.000
min_latency_ns 32800.000
max_latency_ns 891060.000
avg_latency_ns 104784.500
type rfc2544_throughput
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:31:22
start_time 2018-06-05 02:31:33
stop_time 2018-06-05 03:02:56
guest_loopback_app testpmd testpmd
4.2.3.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.51
%system 0.95
%guest 0.00
%CPU 1.47
CPU
minflt/s 0.17
majflt/s 0.00
VSZ 3364228
RSS 8071
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5808
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.04
%system 1.92
%guest 198.17
%CPU 200.13
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 2783674
RSS 68472
%MEM 0.10
kB_rd/s 0.07
kB_wr/s 12.40
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 1.80
%guest 198.27
%CPU 200.09
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2850364
RSS 70796
%MEM 0.11
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.3.7. Anomalies

No anomalies were detected during the course of this test.

4.2.3.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.4. Test ID: PHY2PHY_TPUT_MOD_VLAN

4.2.4.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 3daf5942e2ede8088898a9589d41b797e27b6d4e

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

Below are test details:

  • Test ID: phy2phy_tput_mod_vlan
  • Description: LTD.Throughput.RFC2544.PacketLossRatioFrameModification
  • Deployment: p2p
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
4.2.4.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.4.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 757999.203
tx_rate_mbps Unknown
throughput_rx_mbps 412.352
tx_rate_percent 2.547
throughput_rx_percent 2.668
frame_loss_percent 0.000
min_latency_ns 4100.000
max_latency_ns 1314920.000
avg_latency_ns 27964.000
type rfc2544_throughput
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:42
start_time 2018-06-05 00:23:45
stop_time 2018-06-05 00:51:28
4.2.4.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.48
%system 1.02
%guest 0.00
%CPU 1.48
CPU
minflt/s 0.19
majflt/s 0.00
VSZ 3364112
RSS 7701
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5728
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.4.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.4.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 822160.777
tx_rate_mbps Unknown
throughput_rx_mbps 868.202
tx_rate_percent 4.867
throughput_rx_percent 4.999
frame_loss_percent 0.000
min_latency_ns 4240.000
max_latency_ns 232220.000
avg_latency_ns 29461.000
type rfc2544_throughput
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:42
start_time 2018-06-05 00:23:45
stop_time 2018-06-05 00:51:28
4.2.4.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.48
%system 1.02
%guest 0.00
%CPU 1.48
CPU
minflt/s 0.19
majflt/s 0.00
VSZ 3364112
RSS 7701
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5728
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.4.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.4.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 773908.450
tx_rate_mbps Unknown
throughput_rx_mbps 3194.694
tx_rate_percent 16.469
throughput_rx_percent 16.593
frame_loss_percent 0.000
min_latency_ns 5080.000
max_latency_ns 252380.000
avg_latency_ns 29577.500
type rfc2544_throughput
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:42
start_time 2018-06-05 00:23:45
stop_time 2018-06-05 00:51:28
4.2.4.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.48
%system 1.02
%guest 0.00
%CPU 1.48
CPU
minflt/s 0.19
majflt/s 0.00
VSZ 3364112
RSS 7701
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5728
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.4.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.4.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 801829.912
tx_rate_mbps Unknown
throughput_rx_mbps 6594.249
tx_rate_percent 33.484
throughput_rx_percent 33.613
frame_loss_percent 0.000
min_latency_ns 5840.000
max_latency_ns 424740.000
avg_latency_ns 34292.000
type rfc2544_throughput
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:42
start_time 2018-06-05 00:23:45
stop_time 2018-06-05 00:51:28
4.2.4.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.48
%system 1.02
%guest 0.00
%CPU 1.48
CPU
minflt/s 0.19
majflt/s 0.00
VSZ 3364112
RSS 7701
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5728
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.4.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.4.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 808300.041
tx_rate_mbps Unknown
throughput_rx_mbps 9841.861
tx_rate_percent 49.727
throughput_rx_percent 49.856
frame_loss_percent 0.000
min_latency_ns 7380.000
max_latency_ns 470060.000
avg_latency_ns 55879.500
type rfc2544_throughput
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:42
start_time 2018-06-05 00:23:45
stop_time 2018-06-05 00:51:28
4.2.4.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.48
%system 1.02
%guest 0.00
%CPU 1.48
CPU
minflt/s 0.19
majflt/s 0.00
VSZ 3364112
RSS 7701
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5728
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.4.7. Anomalies

No anomalies were detected during the course of this test.

4.2.4.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.5. Test ID: PVVP_BACK2BACK

4.2.5.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvvp

  • vswitchperf: GIT tag: 3daf5942e2ede8088898a9589d41b797e27b6d4e

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • VNF: QemuVirtioNet, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-16.04_trex_20180209.qcow2
    • /home/jenkins/vloop-vnf-ubuntu-16.04_trex_20180209.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvvp_back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: pvvp
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
4.2.5.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.5.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 425
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:44:26
start_time 2018-06-05 03:02:56
stop_time 2018-06-05 03:47:23
guest_loopback_app testpmd testpmd
4.2.5.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.50
%system 1.02
%guest 0.00
%CPU 1.53
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 3364224
RSS 8076
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5804
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.05
%system 0.17
%guest 199.94
%CPU 200.16
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2848308
RSS 67043
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.62
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.07
%system 0.20
%guest 199.96
%CPU 200.23
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2856532
RSS 68520
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.5.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.5.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 966
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:44:26
start_time 2018-06-05 03:02:56
stop_time 2018-06-05 03:47:23
guest_loopback_app testpmd testpmd
4.2.5.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.50
%system 1.02
%guest 0.00
%CPU 1.53
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 3364224
RSS 8076
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5804
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.05
%system 0.17
%guest 199.94
%CPU 200.16
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2848308
RSS 67043
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.62
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.07
%system 0.20
%guest 199.96
%CPU 200.23
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2856532
RSS 68520
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.5.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.5.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 806
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:44:26
start_time 2018-06-05 03:02:56
stop_time 2018-06-05 03:47:23
guest_loopback_app testpmd testpmd
4.2.5.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.50
%system 1.02
%guest 0.00
%CPU 1.53
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 3364224
RSS 8076
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5804
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.05
%system 0.17
%guest 199.94
%CPU 200.16
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2848308
RSS 67043
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.62
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.07
%system 0.20
%guest 199.96
%CPU 200.23
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2856532
RSS 68520
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.5.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.5.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1096
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:44:26
start_time 2018-06-05 03:02:56
stop_time 2018-06-05 03:47:23
guest_loopback_app testpmd testpmd
4.2.5.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.50
%system 1.02
%guest 0.00
%CPU 1.53
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 3364224
RSS 8076
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5804
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.05
%system 0.17
%guest 199.94
%CPU 200.16
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2848308
RSS 67043
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.62
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.07
%system 0.20
%guest 199.96
%CPU 200.23
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2856532
RSS 68520
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.5.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.5.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 744
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:44:26
start_time 2018-06-05 03:02:56
stop_time 2018-06-05 03:47:23
guest_loopback_app testpmd testpmd
4.2.5.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.50
%system 1.02
%guest 0.00
%CPU 1.53
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 3364224
RSS 8076
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5804
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.05
%system 0.17
%guest 199.94
%CPU 200.16
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2848308
RSS 67043
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.62
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.07
%system 0.20
%guest 199.96
%CPU 200.23
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2856532
RSS 68520
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.5.7. Anomalies

No anomalies were detected during the course of this test.

4.2.5.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.6. Test ID: PVP_BACK2BACK

4.2.6.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: 3daf5942e2ede8088898a9589d41b797e27b6d4e

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • VNF: QemuVirtioNet, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-16.04_trex_20180209.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvp_back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: pvp
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
4.2.6.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.6.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 425
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:38
start_time 2018-06-05 01:47:54
stop_time 2018-06-05 02:31:33
guest_loopback_app testpmd
4.2.6.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.51
%system 1.05
%guest 0.00
%CPU 1.57
CPU
minflt/s 0.15
majflt/s 0.00
VSZ 3364104
RSS 7717
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5820
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.07
%system 0.16
%guest 199.98
%CPU 200.20
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2847047
RSS 67765
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 6.46
kB_ccwr/s 0.00
4.2.6.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.6.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 483
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:38
start_time 2018-06-05 01:47:54
stop_time 2018-06-05 02:31:33
guest_loopback_app testpmd
4.2.6.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.51
%system 1.05
%guest 0.00
%CPU 1.57
CPU
minflt/s 0.15
majflt/s 0.00
VSZ 3364104
RSS 7717
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5820
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.07
%system 0.16
%guest 199.98
%CPU 200.20
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2847047
RSS 67765
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 6.46
kB_ccwr/s 0.00
4.2.6.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.6.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 537
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:38
start_time 2018-06-05 01:47:54
stop_time 2018-06-05 02:31:33
guest_loopback_app testpmd
4.2.6.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.51
%system 1.05
%guest 0.00
%CPU 1.57
CPU
minflt/s 0.15
majflt/s 0.00
VSZ 3364104
RSS 7717
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5820
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.07
%system 0.16
%guest 199.98
%CPU 200.20
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2847047
RSS 67765
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 6.46
kB_ccwr/s 0.00
4.2.6.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.6.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 274
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:38
start_time 2018-06-05 01:47:54
stop_time 2018-06-05 02:31:33
guest_loopback_app testpmd
4.2.6.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.51
%system 1.05
%guest 0.00
%CPU 1.57
CPU
minflt/s 0.15
majflt/s 0.00
VSZ 3364104
RSS 7717
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5820
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.07
%system 0.16
%guest 199.98
%CPU 200.20
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2847047
RSS 67765
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 6.46
kB_ccwr/s 0.00
4.2.6.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.6.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 744
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:38
start_time 2018-06-05 01:47:54
stop_time 2018-06-05 02:31:33
guest_loopback_app testpmd
4.2.6.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.51
%system 1.05
%guest 0.00
%CPU 1.57
CPU
minflt/s 0.15
majflt/s 0.00
VSZ 3364104
RSS 7717
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5820
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.07
%system 0.16
%guest 199.98
%CPU 200.20
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2847047
RSS 67765
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 6.46
kB_ccwr/s 0.00
4.2.6.7. Anomalies

No anomalies were detected during the course of this test.

4.2.6.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.7. Test ID: PHY2PHY_TPUT

4.2.7.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 3daf5942e2ede8088898a9589d41b797e27b6d4e

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

Below are test details:

  • Test ID: phy2phy_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: p2p
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
4.2.7.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.7.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 757999.381
tx_rate_mbps Unknown
throughput_rx_mbps 388.096
tx_rate_percent 2.547
throughput_rx_percent 2.547
frame_loss_percent 0.000
min_latency_ns 4040.000
max_latency_ns 226840.000
avg_latency_ns 29035.000
type rfc2544_throughput
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:42
start_time 2018-06-04 23:13:37
stop_time 2018-06-04 23:41:20
4.2.7.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.49
%system 1.06
%guest 0.00
%CPU 1.54
CPU
minflt/s 0.16
majflt/s 0.00
VSZ 3364104
RSS 7830
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 6004
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.7.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.7.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 952809.447
tx_rate_mbps Unknown
throughput_rx_mbps 975.677
tx_rate_percent 5.641
throughput_rx_percent 5.641
frame_loss_percent 0.000
min_latency_ns 4180.000
max_latency_ns 305900.000
avg_latency_ns 28835.500
type rfc2544_throughput
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:42
start_time 2018-06-04 23:13:37
stop_time 2018-06-04 23:41:20
4.2.7.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.49
%system 1.06
%guest 0.00
%CPU 1.54
CPU
minflt/s 0.16
majflt/s 0.00
VSZ 3364104
RSS 7830
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 6004
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.7.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.7.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 810254.403
tx_rate_mbps Unknown
throughput_rx_mbps 3318.802
tx_rate_percent 17.242
throughput_rx_percent 17.242
frame_loss_percent 0.000
min_latency_ns 5020.000
max_latency_ns 325660.000
avg_latency_ns 29583.500
type rfc2544_throughput
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:42
start_time 2018-06-04 23:13:37
stop_time 2018-06-04 23:41:20
4.2.7.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.49
%system 1.06
%guest 0.00
%CPU 1.54
CPU
minflt/s 0.16
majflt/s 0.00
VSZ 3364104
RSS 7830
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 6004
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.7.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.7.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 838871.224
tx_rate_mbps Unknown
throughput_rx_mbps 6872.033
tx_rate_percent 35.031
throughput_rx_percent 35.031
frame_loss_percent 0.000
min_latency_ns 5760.000
max_latency_ns 413720.000
avg_latency_ns 34457.500
type rfc2544_throughput
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:42
start_time 2018-06-04 23:13:37
stop_time 2018-06-04 23:41:20
4.2.7.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.49
%system 1.06
%guest 0.00
%CPU 1.54
CPU
minflt/s 0.16
majflt/s 0.00
VSZ 3364104
RSS 7830
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 6004
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.7.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.7.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 846015.110
tx_rate_mbps Unknown
throughput_rx_mbps 10274.007
tx_rate_percent 52.047
throughput_rx_percent 52.047
frame_loss_percent 0.000
min_latency_ns 7720.000
max_latency_ns 574540.000
avg_latency_ns 49343.500
type rfc2544_throughput
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:42
start_time 2018-06-04 23:13:37
stop_time 2018-06-04 23:41:20
4.2.7.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.49
%system 1.06
%guest 0.00
%CPU 1.54
CPU
minflt/s 0.16
majflt/s 0.00
VSZ 3364104
RSS 7830
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 6004
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.7.7. Anomalies

No anomalies were detected during the course of this test.

4.2.7.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.8. Test ID: PVP_TPUT

4.2.8.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: 3daf5942e2ede8088898a9589d41b797e27b6d4e

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • VNF: QemuVirtioNet, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-16.04_trex_20180209.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvp_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvp
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
4.2.8.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.8.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 225665.554
tx_rate_mbps Unknown
throughput_rx_mbps 115.541
tx_rate_percent 1.773
throughput_rx_percent 0.758
frame_loss_percent 57.244
min_latency_ns 18540.000
max_latency_ns 83662060.000
avg_latency_ns 5481711.500
type rfc2544_throughput
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:28:46
start_time 2018-06-05 01:19:07
stop_time 2018-06-05 01:47:53
guest_loopback_app testpmd
4.2.8.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.52
%system 1.04
%guest 0.00
%CPU 1.55
CPU
minflt/s 0.24
majflt/s 0.00
VSZ 3364112
RSS 7715
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5884
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.07
%system 1.53
%guest 198.60
%CPU 200.19
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 3031359
RSS 62758
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.10
kB_ccwr/s 0.00
4.2.8.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.8.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 299567.789
tx_rate_mbps Unknown
throughput_rx_mbps 306.757
tx_rate_percent 1.773
throughput_rx_percent 1.773
frame_loss_percent 0.000
min_latency_ns 12420.000
max_latency_ns 1491900.000
avg_latency_ns 54234.000
type rfc2544_throughput
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:28:46
start_time 2018-06-05 01:19:07
stop_time 2018-06-05 01:47:53
guest_loopback_app testpmd
4.2.8.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.52
%system 1.04
%guest 0.00
%CPU 1.55
CPU
minflt/s 0.24
majflt/s 0.00
VSZ 3364112
RSS 7715
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5884
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.07
%system 1.53
%guest 198.60
%CPU 200.19
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 3031359
RSS 62758
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.10
kB_ccwr/s 0.00
4.2.8.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.8.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 228721.527
tx_rate_mbps Unknown
throughput_rx_mbps 936.843
tx_rate_percent 4.867
throughput_rx_percent 4.867
frame_loss_percent 0.000
min_latency_ns 15120.000
max_latency_ns 1239460.000
avg_latency_ns 67844.000
type rfc2544_throughput
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:28:46
start_time 2018-06-05 01:19:07
stop_time 2018-06-05 01:47:53
guest_loopback_app testpmd
4.2.8.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.52
%system 1.04
%guest 0.00
%CPU 1.55
CPU
minflt/s 0.24
majflt/s 0.00
VSZ 3364112
RSS 7715
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5884
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.07
%system 1.53
%guest 198.60
%CPU 200.19
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 3031359
RSS 62758
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.10
kB_ccwr/s 0.00
4.2.8.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.8.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 246198.855
tx_rate_mbps Unknown
throughput_rx_mbps 2016.861
tx_rate_percent 10.281
throughput_rx_percent 10.281
frame_loss_percent 0.000
min_latency_ns 14560.000
max_latency_ns 1288620.000
avg_latency_ns 67698.000
type rfc2544_throughput
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:28:46
start_time 2018-06-05 01:19:07
stop_time 2018-06-05 01:47:53
guest_loopback_app testpmd
4.2.8.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.52
%system 1.04
%guest 0.00
%CPU 1.55
CPU
minflt/s 0.24
majflt/s 0.00
VSZ 3364112
RSS 7715
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5884
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.07
%system 1.53
%guest 198.60
%CPU 200.19
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 3031359
RSS 62758
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.10
kB_ccwr/s 0.00
4.2.8.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.8.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 267698.066
tx_rate_mbps Unknown
throughput_rx_mbps 3250.925
tx_rate_percent 16.469
throughput_rx_percent 16.469
frame_loss_percent 0.000
min_latency_ns 17880.000
max_latency_ns 1239260.000
avg_latency_ns 75231.500
type rfc2544_throughput
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:28:46
start_time 2018-06-05 01:19:07
stop_time 2018-06-05 01:47:53
guest_loopback_app testpmd
4.2.8.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.52
%system 1.04
%guest 0.00
%CPU 1.55
CPU
minflt/s 0.24
majflt/s 0.00
VSZ 3364112
RSS 7715
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5884
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.07
%system 1.53
%guest 198.60
%CPU 200.19
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 3031359
RSS 62758
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.10
kB_ccwr/s 0.00
4.2.8.7. Anomalies

No anomalies were detected during the course of this test.

4.2.8.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.3. Rationale for decisions

The tests conducted do not have pass/fail/conditional-pass criteria. The test is simply conducted and the results are reported.

4.4. Conclusions and recommendations

The test results are stable. The vsperf CI jobs that were used to obtain the results can be found at https://artifacts.opnfv.org/logs/vswitchperf/intel-pod12/2018-06-05_03-05-23/vswitchperf_logs_2018-06-05_03-05-23.tar.gz.

4.5. General

4.5.1. Glossary

  • NFV - Network Function Virtualization
  • Mbps - 1,000,000bps

4.5.2. Document change procedures and history

Document ID Author Date Modified
OPNFV_vswitchperf_LTR_ver_1.0_Jan_15_CN_DRAFT Christopher Nolan 23/01/2015
OPNFV_vswitchperf_LTR_ver_1.1_Jan_15_CN_DRAFT Christopher Nolan 28/01/2015

5. Performance report for VppDpdkVhost

5.1. Introduction

The objective of the OPNFV project titled “Characterise vSwitch Performance for Telco NFV Use Cases”, is to evaluate a virtual switch to identify its suitability for a Telco Network Function Virtualization (NFV) environment. As well as this, the project aims to identify any gaps or bottlenecks in order to drive architectural changes to improve virtual switch performance and determinism. The purpose of this document is to summarize the results of the tests carried out on the virtual switch in the Network Function Virtualization Infrastructure (NFVI) and, from these results, provide evaluations and recommendations for the virtual switch. Test results will be outlined in details-of-LTR, preceded by the document-identifier and the scope and references).

This document is currently in draft form.

5.1.1. Document identifier

The document id will be used to uniquely identify versions of the LTR. The format for the document id will be: OPNFV_vswitchperf_LTR_rel_STATUS, the status is one of: DRAFT, REVIEWED, CORRECTED or FINAL. The document id for this version of the LTR is: OPNFV_vswitchperf_LTR_Brahmaputra_DRAFT.

5.1.2. Scope

The scope of this report is to detail the results of the tests that have been performed on the virtual switch. This report will also evaluate the results of these tests and, based on these evaluations, provide recommendations on the suitability of the virtual switch for use in a Telco NFV environment.

5.2. Details of the Level Test Report

This section provides a test-results-overview. Also included are the rationale and the conclusions.

5.2.1. Test ID: PVP_TPUT_VPP

5.2.1.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: clean

  • vswitchperf: GIT tag: 3daf5942e2ede8088898a9589d41b797e27b6d4e

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: VppDpdkVhost, Version: v17.07-release, GIT tag: NA

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvp_tput_vpp
  • Description: VPP: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: clean
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
5.2.1.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

5.2.1.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 3980655.473
tx_rate_mbps Unknown
throughput_rx_mbps 2038.096
tx_rate_percent 13.375
throughput_rx_percent 13.375
frame_loss_percent 0.000
min_latency_ns 7200.000
max_latency_ns 59180.000
avg_latency_ns 12872.000
type rfc2544_throughput
packet_size 64
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:31:20
start_time 2018-06-05 04:51:50
stop_time 2018-06-05 05:23:10
5.2.1.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.77
%system 0.31
%guest 0.00
%CPU 601.08
CPU
minflt/s 0.08
majflt/s 0.00
VSZ 9861922
RSS 50973
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.1.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

5.2.1.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2259291.250
tx_rate_mbps Unknown
throughput_rx_mbps 2313.514
tx_rate_percent 13.375
throughput_rx_percent 13.375
frame_loss_percent 0.000
min_latency_ns 6180.000
max_latency_ns 35320.000
avg_latency_ns 9470.500
type rfc2544_throughput
packet_size 128
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:31:20
start_time 2018-06-05 04:51:50
stop_time 2018-06-05 05:23:10
5.2.1.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.77
%system 0.31
%guest 0.00
%CPU 601.08
CPU
minflt/s 0.08
majflt/s 0.00
VSZ 9861922
RSS 50973
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.1.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

5.2.1.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2954649.354
tx_rate_mbps Unknown
throughput_rx_mbps 12102.244
tx_rate_percent 62.875
throughput_rx_percent 62.875
frame_loss_percent 0.000
min_latency_ns 9840.000
max_latency_ns 47100.000
avg_latency_ns 17951.500
type rfc2544_throughput
packet_size 512
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:31:20
start_time 2018-06-05 04:51:50
stop_time 2018-06-05 05:23:10
5.2.1.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.77
%system 0.31
%guest 0.00
%CPU 601.08
CPU
minflt/s 0.08
majflt/s 0.00
VSZ 9861922
RSS 50973
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.1.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

5.2.1.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394572.188
tx_rate_mbps Unknown
throughput_rx_mbps 19616.335
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 10160.000
max_latency_ns 94860.000
avg_latency_ns 80406.500
type rfc2544_throughput
packet_size 1024
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:31:20
start_time 2018-06-05 04:51:50
stop_time 2018-06-05 05:23:10
5.2.1.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.77
%system 0.31
%guest 0.00
%CPU 601.08
CPU
minflt/s 0.08
majflt/s 0.00
VSZ 9861922
RSS 50973
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.1.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

5.2.1.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625449.693
tx_rate_mbps Unknown
throughput_rx_mbps 19739.461
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 12280.000
max_latency_ns 89980.000
avg_latency_ns 72266.500
type rfc2544_throughput
packet_size 1518
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:31:20
start_time 2018-06-05 04:51:50
stop_time 2018-06-05 05:23:10
5.2.1.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.77
%system 0.31
%guest 0.00
%CPU 601.08
CPU
minflt/s 0.08
majflt/s 0.00
VSZ 9861922
RSS 50973
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.1.7. Anomalies

No anomalies were detected during the course of this test.

5.2.1.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

5.2.2. Test ID: PVP_BACK2BACK_VPP

5.2.2.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: clean

  • vswitchperf: GIT tag: 3daf5942e2ede8088898a9589d41b797e27b6d4e

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: VppDpdkVhost, Version: v17.07-release, GIT tag: NA

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvp_back2back_vpp
  • Description: VPP: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: clean
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
5.2.2.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

5.2.2.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544_back2back
packet_size 64
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:57:45
start_time 2018-06-05 05:23:11
stop_time 2018-06-05 06:20:56
5.2.2.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.51
%system 0.30
%guest 0.00
%CPU 600.81
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 10026850
RSS 50858
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.2.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

5.2.2.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 966
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 128
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:57:45
start_time 2018-06-05 05:23:11
stop_time 2018-06-05 06:20:56
5.2.2.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.51
%system 0.30
%guest 0.00
%CPU 600.81
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 10026850
RSS 50858
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.2.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

5.2.2.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 70488721
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:57:45
start_time 2018-06-05 05:23:11
stop_time 2018-06-05 06:20:56
5.2.2.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.51
%system 0.30
%guest 0.00
%CPU 600.81
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 10026850
RSS 50858
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.2.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

5.2.2.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 35919540
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:57:45
start_time 2018-06-05 05:23:11
stop_time 2018-06-05 06:20:56
5.2.2.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.51
%system 0.30
%guest 0.00
%CPU 600.81
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 10026850
RSS 50858
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.2.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

5.2.2.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 24382314
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:57:45
start_time 2018-06-05 05:23:11
stop_time 2018-06-05 06:20:56
5.2.2.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.51
%system 0.30
%guest 0.00
%CPU 600.81
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 10026850
RSS 50858
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.2.7. Anomalies

No anomalies were detected during the course of this test.

5.2.2.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

5.2.3. Test ID: PHY2PHY_BACK2BACK_VPP

5.2.3.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: clean

  • vswitchperf: GIT tag: 3daf5942e2ede8088898a9589d41b797e27b6d4e

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: VppDpdkVhost, Version: v17.07-release, GIT tag: NA

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: phy2phy_back2back_vpp
  • Description: VPP: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: clean
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
5.2.3.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

5.2.3.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 23841
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 64
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:37:28
start_time 2018-06-05 04:14:21
stop_time 2018-06-05 04:51:50
5.2.3.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.46
%system 0.30
%guest 0.00
%CPU 600.75
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 3937580
RSS 50862
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.3.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

5.2.3.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 278369
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 128
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:37:28
start_time 2018-06-05 04:14:21
stop_time 2018-06-05 04:51:50
5.2.3.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.46
%system 0.30
%guest 0.00
%CPU 600.75
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 3937580
RSS 50862
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.3.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

5.2.3.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 70488721
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:37:28
start_time 2018-06-05 04:14:21
stop_time 2018-06-05 04:51:50
5.2.3.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.46
%system 0.30
%guest 0.00
%CPU 600.75
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 3937580
RSS 50862
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.3.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

5.2.3.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 35919540
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:37:28
start_time 2018-06-05 04:14:21
stop_time 2018-06-05 04:51:50
5.2.3.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.46
%system 0.30
%guest 0.00
%CPU 600.75
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 3937580
RSS 50862
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.3.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

5.2.3.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 24382314
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:37:28
start_time 2018-06-05 04:14:21
stop_time 2018-06-05 04:51:50
5.2.3.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.46
%system 0.30
%guest 0.00
%CPU 600.75
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 3937580
RSS 50862
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.3.7. Anomalies

No anomalies were detected during the course of this test.

5.2.3.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

5.2.4. Test ID: PHY2PHY_TPUT_VPP

5.2.4.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: clean

  • vswitchperf: GIT tag: 3daf5942e2ede8088898a9589d41b797e27b6d4e

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: VppDpdkVhost, Version: v17.07-release, GIT tag: NA

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: phy2phy_tput_vpp
  • Description: VPP: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: clean
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
5.2.4.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

5.2.4.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 22856315.198
tx_rate_mbps Unknown
throughput_rx_mbps 11702.433
tx_rate_percent 76.797
throughput_rx_percent 76.797
frame_loss_percent 0.000
min_latency_ns 5020.000
max_latency_ns 51860.000
avg_latency_ns 22666.500
type rfc2544_throughput
packet_size 64
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:26:46
start_time 2018-06-05 03:47:34
stop_time 2018-06-05 04:14:21
5.2.4.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.62
%system 0.30
%guest 0.00
%CPU 600.92
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 3937580
RSS 50772
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.4.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

5.2.4.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 16630608.211
tx_rate_mbps Unknown
throughput_rx_mbps 17029.743
tx_rate_percent 98.453
throughput_rx_percent 98.453
frame_loss_percent 0.000
min_latency_ns 5640.000
max_latency_ns 31660.000
avg_latency_ns 12928.500
type rfc2544_throughput
packet_size 128
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:26:46
start_time 2018-06-05 03:47:34
stop_time 2018-06-05 04:14:21
5.2.4.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.62
%system 0.30
%guest 0.00
%CPU 600.92
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 3937580
RSS 50772
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.4.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

5.2.4.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699209.536
tx_rate_mbps Unknown
throughput_rx_mbps 19247.962
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 5500.000
max_latency_ns 30680.000
avg_latency_ns 19586.500
type rfc2544_throughput
packet_size 512
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:26:46
start_time 2018-06-05 03:47:34
stop_time 2018-06-05 04:14:21
5.2.4.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.62
%system 0.30
%guest 0.00
%CPU 600.92
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 3937580
RSS 50772
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.4.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

5.2.4.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394615.705
tx_rate_mbps Unknown
throughput_rx_mbps 19616.692
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 6340.000
max_latency_ns 31920.000
avg_latency_ns 21143.500
type rfc2544_throughput
packet_size 1024
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:26:46
start_time 2018-06-05 03:47:34
stop_time 2018-06-05 04:14:21
5.2.4.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.62
%system 0.30
%guest 0.00
%CPU 600.92
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 3937580
RSS 50772
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.4.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

5.2.4.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625472.866
tx_rate_mbps Unknown
throughput_rx_mbps 19739.742
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 6560.000
max_latency_ns 33980.000
avg_latency_ns 23472.500
type rfc2544_throughput
packet_size 1518
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:26:46
start_time 2018-06-05 03:47:34
stop_time 2018-06-05 04:14:21
5.2.4.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.62
%system 0.30
%guest 0.00
%CPU 600.92
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 3937580
RSS 50772
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.4.7. Anomalies

No anomalies were detected during the course of this test.

5.2.4.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

5.3. Rationale for decisions

The tests conducted do not have pass/fail/conditional-pass criteria. The test is simply conducted and the results are reported.

5.4. Conclusions and recommendations

The test results are stable. The vsperf CI jobs that were used to obtain the results can be found at https://artifacts.opnfv.org/logs/vswitchperf/intel-pod12/2018-06-05_03-05-23/vswitchperf_logs_2018-06-05_03-05-23.tar.gz.

5.5. General

5.5.1. Glossary

  • NFV - Network Function Virtualization
  • Mbps - 1,000,000bps

5.5.2. Document change procedures and history

Document ID Author Date Modified
OPNFV_vswitchperf_LTR_ver_1.0_Jan_15_CN_DRAFT Christopher Nolan 23/01/2015
OPNFV_vswitchperf_LTR_ver_1.1_Jan_15_CN_DRAFT Christopher Nolan 28/01/2015

6. Performance report for none

6.1. Introduction

The objective of the OPNFV project titled “Characterise vSwitch Performance for Telco NFV Use Cases”, is to evaluate a virtual switch to identify its suitability for a Telco Network Function Virtualization (NFV) environment. As well as this, the project aims to identify any gaps or bottlenecks in order to drive architectural changes to improve virtual switch performance and determinism. The purpose of this document is to summarize the results of the tests carried out on the virtual switch in the Network Function Virtualization Infrastructure (NFVI) and, from these results, provide evaluations and recommendations for the virtual switch. Test results will be outlined in details-of-LTR, preceded by the document-identifier and the scope and references).

This document is currently in draft form.

6.1.1. Document identifier

The document id will be used to uniquely identify versions of the LTR. The format for the document id will be: OPNFV_vswitchperf_LTR_rel_STATUS, the status is one of: DRAFT, REVIEWED, CORRECTED or FINAL. The document id for this version of the LTR is: OPNFV_vswitchperf_LTR_Brahmaputra_DRAFT.

6.1.2. Scope

The scope of this report is to detail the results of the tests that have been performed on the virtual switch. This report will also evaluate the results of these tests and, based on these evaluations, provide recommendations on the suitability of the virtual switch for use in a Telco NFV environment.

6.2. Details of the Level Test Report

This section provides a test-results-overview. Also included are the rationale and the conclusions.

6.2.1. Test ID: PVP_TPUT

6.2.1.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
    • Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: 3daf5942e2ede8088898a9589d41b797e27b6d4e

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: none, Version: None, GIT tag: None

  • VNF: QemuPciPassthrough, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvp_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvp
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
6.2.1.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

6.2.1.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 988188.454
tx_rate_mbps Unknown
throughput_rx_mbps 505.952
tx_rate_percent 3.320
throughput_rx_percent 3.320
frame_loss_percent 0.000
min_latency_ns 3440.000
max_latency_ns 177000.000
avg_latency_ns 3580.500
type rfc2544_throughput
packet_size 64
vswitch none
traffic_type udp
test_execution_time 00:28:13
start_time 2018-06-05 06:32:32
stop_time 2018-06-05 07:00:45
guest_loopback_app testpmd
6.2.1.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.13
%system 0.21
%guest 200.09
%CPU 200.43
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2820970
RSS 75291
%MEM 0.11
kB_rd/s 0.00
kB_wr/s 13.28
kB_ccwr/s 0.00
6.2.1.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

6.2.1.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1083456.641
tx_rate_mbps Unknown
throughput_rx_mbps 1109.460
tx_rate_percent 6.414
throughput_rx_percent 6.414
frame_loss_percent 0.000
min_latency_ns 3540.000
max_latency_ns 23800.000
avg_latency_ns 3707.500
type rfc2544_throughput
packet_size 128
vswitch none
traffic_type udp
test_execution_time 00:28:13
start_time 2018-06-05 06:32:32
stop_time 2018-06-05 07:00:45
guest_loopback_app testpmd
6.2.1.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.13
%system 0.21
%guest 200.09
%CPU 200.43
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2820970
RSS 75291
%MEM 0.11
kB_rd/s 0.00
kB_wr/s 13.28
kB_ccwr/s 0.00
6.2.1.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

6.2.1.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699198.838
tx_rate_mbps Unknown
throughput_rx_mbps 19247.918
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 5060.000
max_latency_ns 37580.000
avg_latency_ns 23698.000
type rfc2544_throughput
packet_size 512
vswitch none
traffic_type udp
test_execution_time 00:28:13
start_time 2018-06-05 06:32:32
stop_time 2018-06-05 07:00:45
guest_loopback_app testpmd
6.2.1.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.13
%system 0.21
%guest 200.09
%CPU 200.43
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2820970
RSS 75291
%MEM 0.11
kB_rd/s 0.00
kB_wr/s 13.28
kB_ccwr/s 0.00
6.2.1.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

6.2.1.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394612.361
tx_rate_mbps Unknown
throughput_rx_mbps 19616.664
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 5880.000
max_latency_ns 38420.000
avg_latency_ns 23219.500
type rfc2544_throughput
packet_size 1024
vswitch none
traffic_type udp
test_execution_time 00:28:13
start_time 2018-06-05 06:32:32
stop_time 2018-06-05 07:00:45
guest_loopback_app testpmd
6.2.1.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.13
%system 0.21
%guest 200.09
%CPU 200.43
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2820970
RSS 75291
%MEM 0.11
kB_rd/s 0.00
kB_wr/s 13.28
kB_ccwr/s 0.00
6.2.1.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

6.2.1.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625467.833
tx_rate_mbps Unknown
throughput_rx_mbps 19739.681
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 6320.000
max_latency_ns 43680.000
avg_latency_ns 31882.500
type rfc2544_throughput
packet_size 1518
vswitch none
traffic_type udp
test_execution_time 00:28:13
start_time 2018-06-05 06:32:32
stop_time 2018-06-05 07:00:45
guest_loopback_app testpmd
6.2.1.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.13
%system 0.21
%guest 200.09
%CPU 200.43
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2820970
RSS 75291
%MEM 0.11
kB_rd/s 0.00
kB_wr/s 13.28
kB_ccwr/s 0.00
6.2.1.7. Anomalies

No anomalies were detected during the course of this test.

6.2.1.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

6.3. Rationale for decisions

The tests conducted do not have pass/fail/conditional-pass criteria. The test is simply conducted and the results are reported.

6.4. Conclusions and recommendations

The test results are stable. The vsperf CI jobs that were used to obtain the results can be found at https://artifacts.opnfv.org/logs/vswitchperf/intel-pod12/2018-06-05_03-05-23/vswitchperf_logs_2018-06-05_03-05-23.tar.gz.

6.5. General

6.5.1. Glossary

  • NFV - Network Function Virtualization
  • Mbps - 1,000,000bps

6.5.2. Document change procedures and history

Document ID Author Date Modified
OPNFV_vswitchperf_LTR_ver_1.0_Jan_15_CN_DRAFT Christopher Nolan 23/01/2015
OPNFV_vswitchperf_LTR_ver_1.1_Jan_15_CN_DRAFT Christopher Nolan 28/01/2015