VSPERF Results

1. VSPERF Test Scenarios

Predefined Tests suitable for automated execution with CI:

Test Definition
phy2phy_tput PacketLossRatio for Phy2Phy
back2back BackToBackFrames for Phy2Phy
phy2phy_tput_mod_vlan PacketLossRatioFrameModification for Phy2Phy
phy2phy_cont Phy2Phy blast vswitch at x% TX rate and measure throughput
pvp_cont PVP blast vswitch at x% TX rate and measure throughput
pvvp_cont PVVP blast vswitch at x% TX rate and measure throughput
phy2phy_scalability Scalability0PacketLoss for Phy2Phy
pvp_tput PacketLossRatio for PVP
pvp_back2back BackToBackFrames for PVP
pvvp_tput PacketLossRatio for PVVP
pvvp_back2back BackToBackFrames for PVVP
phy2phy_cpu_load CPU0PacketLoss for Phy2Phy
phy2phy_mem_load Same as CPU0PacketLoss but using a memory intensive app

Deployment topologies:

  • Phy2Phy: Physical port -> vSwitch -> Physical port.
  • PVP: Physical port -> vSwitch -> VNF -> vSwitch -> Physical port.
  • PVVP: Physical port -> vSwitch -> VNF -> vSwitch -> VNF -> vSwitch -> Physical port.

Loopback applications in the Guest:

Supported traffic generators:

  • Spirent Testcenter
  • Ixia: IxOS and IxNet.
  • Xena
  • MoonGen
  • Dummy
  • T-Rex

2. OPNFV Test Results

VSPERF CI jobs are run daily and sample results can be found at https://wiki.opnfv.org/display/vsperf/Vsperf+Results

Testcase names shown in the dashboard are combination of orignal testcase name from VSPERF framework and indication of used vswitch.

Example:

Testcase phy2phy_tput is executed for three vSwitch types: OvsDpdkVhost, OvsVanilla and VppDpdkVhost. In this case, following testcase names will be used in the dashboard: phy2phy_tput_ovsdpdkvhost, phy2phy_tput_ovsvanilla and phy2phy_tput_vppdpdkvhost.

In case of RFC2544 Throughput test, the recorded metric is FPS (frames per second) without packet loss. For RFC2544 Back2Back test, the recorded metric is back-to-back value (number of frames) without packet loss.

The loopback application in the VNF used for PVP and PVVP scenarios was DPDK testpmd.

Guest interface types are vhost-user for OvsDpdkVhost and VppDpdkVhost and virtio-net for OvsVanilla.

3. Performance report for OvsDpdkVhost

3.1. Introduction

The objective of the OPNFV project titled “Characterise vSwitch Performance for Telco NFV Use Cases”, is to evaluate a virtual switch to identify its suitability for a Telco Network Function Virtualization (NFV) environment. As well as this, the project aims to identify any gaps or bottlenecks in order to drive architectural changes to improve virtual switch performance and determinism. The purpose of this document is to summarize the results of the tests carried out on the virtual switch in the Network Function Virtualization Infrastructure (NFVI) and, from these results, provide evaluations and recommendations for the virtual switch. Test results will be outlined in details-of-LTR, preceded by the document-identifier and the scope and references).

This document is currently in draft form.

3.1.1. Document identifier

The document id will be used to uniquely identify versions of the LTR. The format for the document id will be: OPNFV_vswitchperf_LTR_rel_STATUS, the status is one of: DRAFT, REVIEWED, CORRECTED or FINAL. The document id for this version of the LTR is: OPNFV_vswitchperf_LTR_Brahmaputra_DRAFT.

3.1.2. Scope

The scope of this report is to detail the results of the tests that have been performed on the virtual switch. This report will also evaluate the results of these tests and, based on these evaluations, provide recommendations on the suitability of the virtual switch for use in a Telco NFV environment.

3.2. Details of the Level Test Report

This section provides a test-results-overview. Also included are the rationale and the conclusions.

3.2.1. Test ID: BACK2BACK

3.2.1.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 998842df061caf3f90adf756c78667262525a6e0

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: p2p
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
3.2.1.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.1.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 21713
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 64
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:33:53
start_time 2018-04-24 10:41:49
stop_time 2018-04-24 11:15:43
3.2.1.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 6811
%usr 100.00
%system 0.06
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056776
RSS 43023
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 6792
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50672
RSS 6660
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.1.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 253378378
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 128
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:33:53
start_time 2018-04-24 10:41:49
stop_time 2018-04-24 11:15:43
3.2.1.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 6811
%usr 100.00
%system 0.06
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056776
RSS 43023
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 6792
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50672
RSS 6660
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.1.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 70488721
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:33:53
start_time 2018-04-24 10:41:49
stop_time 2018-04-24 11:15:43
3.2.1.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 6811
%usr 100.00
%system 0.06
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056776
RSS 43023
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 6792
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50672
RSS 6660
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.1.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 35919540
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:33:53
start_time 2018-04-24 10:41:49
stop_time 2018-04-24 11:15:43
3.2.1.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 6811
%usr 100.00
%system 0.06
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056776
RSS 43023
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 6792
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50672
RSS 6660
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.1.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 24382314
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:33:53
start_time 2018-04-24 10:41:49
stop_time 2018-04-24 11:15:43
3.2.1.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 6811
%usr 100.00
%system 0.06
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056776
RSS 43023
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 6792
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50672
RSS 6660
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.7. Anomalies

No anomalies were detected during the course of this test.

3.2.1.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.2. Test ID: PHY2PHY_SCALABILITY

3.2.2.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 998842df061caf3f90adf756c78667262525a6e0

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: phy2phy_scalability
  • Description: LTD.Scalability.Flows.RFC2544.0PacketLoss
  • Deployment: p2p
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
3.2.2.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.2.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 15720370.541
tx_rate_mbps Unknown
throughput_rx_mbps 8048.830
tx_rate_percent 52.820
throughput_rx_percent 52.820
frame_loss_percent 0.000
min_latency_ns 5360.000
max_latency_ns 83760.000
avg_latency_ns 13907.000
type rfc2544_throughput
packet_size 64
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:19:45
start_time 2018-04-24 11:44:32
stop_time 2018-04-24 12:04:17
stream_count 8000
match_type L4
pre-installed_flows No
3.2.2.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 10894
%usr 100.00
%system 0.06
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 5056772
RSS 43026
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 10876
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50668
RSS 6676
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.2.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.2.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 15716123.032
tx_rate_mbps Unknown
throughput_rx_mbps 16093.310
tx_rate_percent 93.039
throughput_rx_percent 93.039
frame_loss_percent 0.000
min_latency_ns 5620.000
max_latency_ns 83900.000
avg_latency_ns 16044.000
type rfc2544_throughput
packet_size 128
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:19:45
start_time 2018-04-24 11:44:32
stop_time 2018-04-24 12:04:17
stream_count 8000
match_type L4
pre-installed_flows No
3.2.2.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 10894
%usr 100.00
%system 0.06
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 5056772
RSS 43026
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 10876
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50668
RSS 6676
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.2.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.2.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699201.125
tx_rate_mbps Unknown
throughput_rx_mbps 19247.928
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 15580.000
max_latency_ns 48900.000
avg_latency_ns 36051.500
type rfc2544_throughput
packet_size 512
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:19:45
start_time 2018-04-24 11:44:32
stop_time 2018-04-24 12:04:17
stream_count 8000
match_type L4
pre-installed_flows No
3.2.2.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 10894
%usr 100.00
%system 0.06
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 5056772
RSS 43026
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 10876
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50668
RSS 6676
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.2.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.2.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394615.634
tx_rate_mbps Unknown
throughput_rx_mbps 19616.691
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 16400.000
max_latency_ns 45940.000
avg_latency_ns 32313.500
type rfc2544_throughput
packet_size 1024
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:19:45
start_time 2018-04-24 11:44:32
stop_time 2018-04-24 12:04:17
stream_count 8000
match_type L4
pre-installed_flows No
3.2.2.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 10894
%usr 100.00
%system 0.06
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 5056772
RSS 43026
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 10876
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50668
RSS 6676
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.2.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.2.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625474.378
tx_rate_mbps Unknown
throughput_rx_mbps 19739.761
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 16840.000
max_latency_ns 44480.000
avg_latency_ns 31512.500
type rfc2544_throughput
packet_size 1518
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:19:45
start_time 2018-04-24 11:44:32
stop_time 2018-04-24 12:04:17
stream_count 8000
match_type L4
pre-installed_flows No
3.2.2.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 10894
%usr 100.00
%system 0.06
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 5056772
RSS 43026
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 10876
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50668
RSS 6676
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.2.7. Anomalies

No anomalies were detected during the course of this test.

3.2.2.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.3. Test ID: PVVP_TPUT

3.2.3.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvvp

  • vswitchperf: GIT tag: 998842df061caf3f90adf756c78667262525a6e0

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

  • VNF: QemuDpdkVhostUser, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvvp_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvvp
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
3.2.3.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.3.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1218378.106
tx_rate_mbps Unknown
throughput_rx_mbps 623.810
tx_rate_percent 4.094
throughput_rx_percent 4.094
frame_loss_percent 0.000
min_latency_ns 9480.000
max_latency_ns 229980.000
avg_latency_ns 16942.500
type rfc2544_throughput
packet_size 64
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:29:24
start_time 2018-04-24 12:56:09
stop_time 2018-04-24 13:25:34
guest_loopback_app testpmd testpmd
3.2.3.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 16066
%usr 100.00
%system 0.08
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 17713420
RSS 43116
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 16047
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50668
RSS 6624
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 16298
%usr 0.03
%system 0.02
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2828440
RSS 66859
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.27
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 16225
%usr 0.08
%system 0.06
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2803944
RSS 66624
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
3.2.3.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.3.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1214105.089
tx_rate_mbps Unknown
throughput_rx_mbps 1243.244
tx_rate_percent 7.188
throughput_rx_percent 7.188
frame_loss_percent 0.000
min_latency_ns 9600.000
max_latency_ns 221660.000
avg_latency_ns 17233.500
type rfc2544_throughput
packet_size 128
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:29:24
start_time 2018-04-24 12:56:09
stop_time 2018-04-24 13:25:34
guest_loopback_app testpmd testpmd
3.2.3.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 16066
%usr 100.00
%system 0.08
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 17713420
RSS 43116
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 16047
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50668
RSS 6624
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 16298
%usr 0.03
%system 0.02
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2828440
RSS 66859
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.27
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 16225
%usr 0.08
%system 0.06
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2803944
RSS 66624
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
3.2.3.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.3.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1246402.540
tx_rate_mbps Unknown
throughput_rx_mbps 5105.265
tx_rate_percent 26.523
throughput_rx_percent 26.523
frame_loss_percent 0.000
min_latency_ns 10840.000
max_latency_ns 219140.000
avg_latency_ns 18637.000
type rfc2544_throughput
packet_size 512
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:29:24
start_time 2018-04-24 12:56:09
stop_time 2018-04-24 13:25:34
guest_loopback_app testpmd testpmd
3.2.3.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 16066
%usr 100.00
%system 0.08
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 17713420
RSS 43116
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 16047
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50668
RSS 6624
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 16298
%usr 0.03
%system 0.02
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2828440
RSS 66859
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.27
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 16225
%usr 0.08
%system 0.06
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2803944
RSS 66624
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
3.2.3.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.3.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1301896.342
tx_rate_mbps Unknown
throughput_rx_mbps 10665.135
tx_rate_percent 54.367
throughput_rx_percent 54.367
frame_loss_percent 0.000
min_latency_ns 12200.000
max_latency_ns 223660.000
avg_latency_ns 20510.000
type rfc2544_throughput
packet_size 1024
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:29:24
start_time 2018-04-24 12:56:09
stop_time 2018-04-24 13:25:34
guest_loopback_app testpmd testpmd
3.2.3.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 16066
%usr 100.00
%system 0.08
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 17713420
RSS 43116
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 16047
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50668
RSS 6624
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 16298
%usr 0.03
%system 0.02
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2828440
RSS 66859
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.27
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 16225
%usr 0.08
%system 0.06
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2803944
RSS 66624
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
3.2.3.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.3.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1298612.405
tx_rate_mbps Unknown
throughput_rx_mbps 15770.349
tx_rate_percent 79.891
throughput_rx_percent 79.891
frame_loss_percent 0.000
min_latency_ns 16420.000
max_latency_ns 224440.000
avg_latency_ns 23067.500
type rfc2544_throughput
packet_size 1518
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:29:24
start_time 2018-04-24 12:56:09
stop_time 2018-04-24 13:25:34
guest_loopback_app testpmd testpmd
3.2.3.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 16066
%usr 100.00
%system 0.08
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 17713420
RSS 43116
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 16047
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50668
RSS 6624
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 16298
%usr 0.03
%system 0.02
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2828440
RSS 66859
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.27
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 16225
%usr 0.08
%system 0.06
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2803944
RSS 66624
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
3.2.3.7. Anomalies

No anomalies were detected during the course of this test.

3.2.3.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.4. Test ID: PHY2PHY_TPUT_MOD_VLAN

3.2.4.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 998842df061caf3f90adf756c78667262525a6e0

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: phy2phy_tput_mod_vlan
  • Description: LTD.Throughput.RFC2544.PacketLossRatioFrameModification
  • Deployment: p2p
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
3.2.4.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.4.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 14569458.292
tx_rate_mbps Unknown
throughput_rx_mbps 7925.785
tx_rate_percent 48.953
throughput_rx_percent 51.284
frame_loss_percent 0.000
min_latency_ns 5680.000
max_latency_ns 35820.000
avg_latency_ns 12681.000
type rfc2544_throughput
packet_size 64
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:28:47
start_time 2018-04-24 11:15:44
stop_time 2018-04-24 11:44:31
3.2.4.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 8967
%usr 100.00
%system 0.06
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056772
RSS 42639
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 8948
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50676
RSS 6508
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.4.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.4.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 14540245.578
tx_rate_mbps Unknown
throughput_rx_mbps 15354.499
tx_rate_percent 86.078
throughput_rx_percent 88.405
frame_loss_percent 0.000
min_latency_ns 6020.000
max_latency_ns 59160.000
avg_latency_ns 15380.500
type rfc2544_throughput
packet_size 128
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:28:47
start_time 2018-04-24 11:15:44
stop_time 2018-04-24 11:44:31
3.2.4.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 8967
%usr 100.00
%system 0.06
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056772
RSS 42639
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 8948
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50676
RSS 6508
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.4.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.4.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4662904.057
tx_rate_mbps Unknown
throughput_rx_mbps 19248.468
tx_rate_percent 99.227
throughput_rx_percent 99.973
frame_loss_percent 0.000
min_latency_ns 5940.000
max_latency_ns 23380.000
avg_latency_ns 10862.000
type rfc2544_throughput
packet_size 512
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:28:47
start_time 2018-04-24 11:15:44
stop_time 2018-04-24 11:44:31
3.2.4.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 8967
%usr 100.00
%system 0.06
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056772
RSS 42639
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 8948
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50676
RSS 6508
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.4.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.4.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2376116.852
tx_rate_mbps Unknown
throughput_rx_mbps 19541.185
tx_rate_percent 99.227
throughput_rx_percent 99.607
frame_loss_percent 0.000
min_latency_ns 6580.000
max_latency_ns 26740.000
avg_latency_ns 9350.500
type rfc2544_throughput
packet_size 1024
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:28:47
start_time 2018-04-24 11:15:44
stop_time 2018-04-24 11:44:31
3.2.4.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 8967
%usr 100.00
%system 0.06
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056772
RSS 42639
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 8948
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50676
RSS 6508
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.4.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.4.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1612916.331
tx_rate_mbps Unknown
throughput_rx_mbps 19638.869
tx_rate_percent 99.227
throughput_rx_percent 99.485
frame_loss_percent 0.000
min_latency_ns 7100.000
max_latency_ns 24580.000
avg_latency_ns 9509.500
type rfc2544_throughput
packet_size 1518
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:28:47
start_time 2018-04-24 11:15:44
stop_time 2018-04-24 11:44:31
3.2.4.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 8967
%usr 100.00
%system 0.06
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056772
RSS 42639
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 8948
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50676
RSS 6508
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.4.7. Anomalies

No anomalies were detected during the course of this test.

3.2.4.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.5. Test ID: PVVP_BACK2BACK

3.2.5.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvvp

  • vswitchperf: GIT tag: 998842df061caf3f90adf756c78667262525a6e0

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

  • VNF: QemuDpdkVhostUser, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvvp_back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: pvvp
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
3.2.5.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.5.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 851
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 64
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:35:51
start_time 2018-04-24 13:25:35
stop_time 2018-04-24 14:01:26
guest_loopback_app testpmd testpmd
3.2.5.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 18190
%usr 100.00
%system 0.07
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 17713420
RSS 43052
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 18161
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50668
RSS 6556
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 18419
%usr 0.02
%system 0.02
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2814224
RSS 68255
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 9.40
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 18349
%usr 0.17
%system 0.11
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2816280
RSS 67572
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.02
kB_ccwr/s 0.00
3.2.5.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.5.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 483
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 128
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:35:51
start_time 2018-04-24 13:25:35
stop_time 2018-04-24 14:01:26
guest_loopback_app testpmd testpmd
3.2.5.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 18190
%usr 100.00
%system 0.07
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 17713420
RSS 43052
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 18161
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50668
RSS 6556
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 18419
%usr 0.02
%system 0.02
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2814224
RSS 68255
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 9.40
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 18349
%usr 0.17
%system 0.11
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2816280
RSS 67572
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.02
kB_ccwr/s 0.00
3.2.5.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.5.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 70488721
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:35:51
start_time 2018-04-24 13:25:35
stop_time 2018-04-24 14:01:26
guest_loopback_app testpmd testpmd
3.2.5.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 18190
%usr 100.00
%system 0.07
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 17713420
RSS 43052
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 18161
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50668
RSS 6556
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 18419
%usr 0.02
%system 0.02
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2814224
RSS 68255
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 9.40
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 18349
%usr 0.17
%system 0.11
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2816280
RSS 67572
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.02
kB_ccwr/s 0.00
3.2.5.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.5.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 35919540
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:35:51
start_time 2018-04-24 13:25:35
stop_time 2018-04-24 14:01:26
guest_loopback_app testpmd testpmd
3.2.5.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 18190
%usr 100.00
%system 0.07
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 17713420
RSS 43052
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 18161
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50668
RSS 6556
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 18419
%usr 0.02
%system 0.02
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2814224
RSS 68255
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 9.40
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 18349
%usr 0.17
%system 0.11
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2816280
RSS 67572
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.02
kB_ccwr/s 0.00
3.2.5.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.5.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 24382314
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:35:51
start_time 2018-04-24 13:25:35
stop_time 2018-04-24 14:01:26
guest_loopback_app testpmd testpmd
3.2.5.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 18190
%usr 100.00
%system 0.07
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 17713420
RSS 43052
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 18161
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50668
RSS 6556
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 18419
%usr 0.02
%system 0.02
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2814224
RSS 68255
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 9.40
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 18349
%usr 0.17
%system 0.11
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2816280
RSS 67572
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.02
kB_ccwr/s 0.00
3.2.5.7. Anomalies

No anomalies were detected during the course of this test.

3.2.5.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.6. Test ID: PVP_BACK2BACK

3.2.6.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: 998842df061caf3f90adf756c78667262525a6e0

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

  • VNF: QemuDpdkVhostUser, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvp_back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: pvp
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
3.2.6.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.6.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 2980
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 64
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:30:23
start_time 2018-04-24 12:25:45
stop_time 2018-04-24 12:56:09
guest_loopback_app testpmd
3.2.6.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 14007
%usr 100.00
%system 0.06
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 11421964
RSS 43180
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 13988
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50668
RSS 6252
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 14154
%usr 0.05
%system 0.03
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2802767
RSS 67004
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 11.28
kB_ccwr/s 0.00
3.2.6.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.6.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1449
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 128
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:30:23
start_time 2018-04-24 12:25:45
stop_time 2018-04-24 12:56:09
guest_loopback_app testpmd
3.2.6.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 14007
%usr 100.00
%system 0.06
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 11421964
RSS 43180
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 13988
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50668
RSS 6252
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 14154
%usr 0.05
%system 0.03
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2802767
RSS 67004
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 11.28
kB_ccwr/s 0.00
3.2.6.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.6.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 70488721
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:30:23
start_time 2018-04-24 12:25:45
stop_time 2018-04-24 12:56:09
guest_loopback_app testpmd
3.2.6.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 14007
%usr 100.00
%system 0.06
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 11421964
RSS 43180
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 13988
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50668
RSS 6252
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 14154
%usr 0.05
%system 0.03
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2802767
RSS 67004
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 11.28
kB_ccwr/s 0.00
3.2.6.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.6.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 35919540
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:30:23
start_time 2018-04-24 12:25:45
stop_time 2018-04-24 12:56:09
guest_loopback_app testpmd
3.2.6.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 14007
%usr 100.00
%system 0.06
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 11421964
RSS 43180
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 13988
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50668
RSS 6252
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 14154
%usr 0.05
%system 0.03
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2802767
RSS 67004
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 11.28
kB_ccwr/s 0.00
3.2.6.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.6.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 24382314
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:30:23
start_time 2018-04-24 12:25:45
stop_time 2018-04-24 12:56:09
guest_loopback_app testpmd
3.2.6.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 14007
%usr 100.00
%system 0.06
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 11421964
RSS 43180
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 13988
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50668
RSS 6252
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 14154
%usr 0.05
%system 0.03
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2802767
RSS 67004
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 11.28
kB_ccwr/s 0.00
3.2.6.7. Anomalies

No anomalies were detected during the course of this test.

3.2.6.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.7. Test ID: PHY2PHY_TPUT

3.2.7.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 998842df061caf3f90adf756c78667262525a6e0

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: phy2phy_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: p2p
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
3.2.7.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.7.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 19633555.452
tx_rate_mbps Unknown
throughput_rx_mbps 10052.380
tx_rate_percent 65.969
throughput_rx_percent 65.969
frame_loss_percent 0.000
min_latency_ns 5180.000
max_latency_ns 28220.000
avg_latency_ns 10369.000
type rfc2544_throughput
packet_size 64
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:16:38
start_time 2018-04-24 10:25:10
stop_time 2018-04-24 10:41:49
3.2.7.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 5484
%usr 100.00
%system 0.06
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 5056776
RSS 43026
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 5465
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50676
RSS 6644
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.7.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.7.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 16891650.065
tx_rate_mbps Unknown
throughput_rx_mbps 17297.050
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 14580.000
max_latency_ns 69280.000
avg_latency_ns 48169.000
type rfc2544_throughput
packet_size 128
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:16:38
start_time 2018-04-24 10:25:10
stop_time 2018-04-24 10:41:49
3.2.7.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 5484
%usr 100.00
%system 0.06
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 5056776
RSS 43026
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 5465
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50676
RSS 6644
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.7.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.7.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699203.944
tx_rate_mbps Unknown
throughput_rx_mbps 19247.939
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 15040.000
max_latency_ns 44820.000
avg_latency_ns 33757.000
type rfc2544_throughput
packet_size 512
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:16:38
start_time 2018-04-24 10:25:10
stop_time 2018-04-24 10:41:49
3.2.7.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 5484
%usr 100.00
%system 0.06
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 5056776
RSS 43026
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 5465
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50676
RSS 6644
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.7.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.7.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394614.532
tx_rate_mbps Unknown
throughput_rx_mbps 19616.682
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 15100.000
max_latency_ns 44020.000
avg_latency_ns 32600.500
type rfc2544_throughput
packet_size 1024
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:16:38
start_time 2018-04-24 10:25:10
stop_time 2018-04-24 10:41:49
3.2.7.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 5484
%usr 100.00
%system 0.06
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 5056776
RSS 43026
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 5465
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50676
RSS 6644
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.7.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.7.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625474.757
tx_rate_mbps Unknown
throughput_rx_mbps 19739.765
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 15400.000
max_latency_ns 40620.000
avg_latency_ns 29805.000
type rfc2544_throughput
packet_size 1518
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:16:38
start_time 2018-04-24 10:25:10
stop_time 2018-04-24 10:41:49
3.2.7.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 5484
%usr 100.00
%system 0.06
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 5056776
RSS 43026
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 5465
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50676
RSS 6644
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.7.7. Anomalies

No anomalies were detected during the course of this test.

3.2.7.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.8. Test ID: PVP_TPUT

3.2.8.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: 998842df061caf3f90adf756c78667262525a6e0

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

  • VNF: QemuDpdkVhostUser, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvp_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvp
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
3.2.8.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.8.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 5591985.734
tx_rate_mbps Unknown
throughput_rx_mbps 2863.097
tx_rate_percent 18.789
throughput_rx_percent 18.789
frame_loss_percent 0.000
min_latency_ns 7820.000
max_latency_ns 61380.000
avg_latency_ns 10712.500
type rfc2544_throughput
packet_size 64
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:21:26
start_time 2018-04-24 12:04:17
stop_time 2018-04-24 12:25:44
guest_loopback_app testpmd
3.2.8.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 12360
%usr 100.00
%system 0.06
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 11421964
RSS 43028
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 12341
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50668
RSS 6508
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 12507
%usr 0.04
%system 0.03
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 2840885
RSS 67535
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 17.46
kB_ccwr/s 0.00
3.2.8.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.8.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 6440037.841
tx_rate_mbps Unknown
throughput_rx_mbps 6594.599
tx_rate_percent 38.125
throughput_rx_percent 38.125
frame_loss_percent 0.000
min_latency_ns 8720.000
max_latency_ns 51000.000
avg_latency_ns 12493.000
type rfc2544_throughput
packet_size 128
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:21:26
start_time 2018-04-24 12:04:17
stop_time 2018-04-24 12:25:44
guest_loopback_app testpmd
3.2.8.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 12360
%usr 100.00
%system 0.06
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 11421964
RSS 43028
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 12341
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50668
RSS 6508
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 12507
%usr 0.04
%system 0.03
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 2840885
RSS 67535
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 17.46
kB_ccwr/s 0.00
3.2.8.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.8.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699176.564
tx_rate_mbps Unknown
throughput_rx_mbps 19247.827
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 25820.000
max_latency_ns 76260.000
avg_latency_ns 62200.500
type rfc2544_throughput
packet_size 512
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:21:26
start_time 2018-04-24 12:04:17
stop_time 2018-04-24 12:25:44
guest_loopback_app testpmd
3.2.8.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 12360
%usr 100.00
%system 0.06
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 11421964
RSS 43028
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 12341
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50668
RSS 6508
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 12507
%usr 0.04
%system 0.03
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 2840885
RSS 67535
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 17.46
kB_ccwr/s 0.00
3.2.8.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.8.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394588.567
tx_rate_mbps Unknown
throughput_rx_mbps 19616.470
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 50360.000
max_latency_ns 112080.000
avg_latency_ns 100059.500
type rfc2544_throughput
packet_size 1024
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:21:26
start_time 2018-04-24 12:04:17
stop_time 2018-04-24 12:25:44
guest_loopback_app testpmd
3.2.8.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 12360
%usr 100.00
%system 0.06
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 11421964
RSS 43028
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 12341
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50668
RSS 6508
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 12507
%usr 0.04
%system 0.03
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 2840885
RSS 67535
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 17.46
kB_ccwr/s 0.00
3.2.8.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.8.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625463.135
tx_rate_mbps Unknown
throughput_rx_mbps 19739.624
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 29640.000
max_latency_ns 76520.000
avg_latency_ns 61640.000
type rfc2544_throughput
packet_size 1518
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:21:26
start_time 2018-04-24 12:04:17
stop_time 2018-04-24 12:25:44
guest_loopback_app testpmd
3.2.8.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 12360
%usr 100.00
%system 0.06
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 11421964
RSS 43028
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 12341
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50668
RSS 6508
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 12507
%usr 0.04
%system 0.03
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 2840885
RSS 67535
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 17.46
kB_ccwr/s 0.00
3.2.8.7. Anomalies

No anomalies were detected during the course of this test.

3.2.8.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.3. Rationale for decisions

The tests conducted do not have pass/fail/conditional-pass criteria. The test is simply conducted and the results are reported.

3.4. Conclusions and recommendations

The test results are stable. The vsperf CI jobs that were used to obtain the results can be found at https://artifacts.opnfv.org/logs/vswitchperf/intel-pod12/fraser/2018-04-24_17-25-03/vswitchperf_logs_2018-04-24_17-25-03.tar.gz.

3.5. General

3.5.1. Glossary

  • NFV - Network Function Virtualization
  • Mbps - 1,000,000bps

3.5.2. Document change procedures and history

Document ID Author Date Modified
OPNFV_vswitchperf_LTR_ver_1.0_Jan_15_CN_DRAFT Christopher Nolan 23/01/2015
OPNFV_vswitchperf_LTR_ver_1.1_Jan_15_CN_DRAFT Christopher Nolan 28/01/2015

4. Performance report for OvsVanilla

4.1. Introduction

The objective of the OPNFV project titled “Characterise vSwitch Performance for Telco NFV Use Cases”, is to evaluate a virtual switch to identify its suitability for a Telco Network Function Virtualization (NFV) environment. As well as this, the project aims to identify any gaps or bottlenecks in order to drive architectural changes to improve virtual switch performance and determinism. The purpose of this document is to summarize the results of the tests carried out on the virtual switch in the Network Function Virtualization Infrastructure (NFVI) and, from these results, provide evaluations and recommendations for the virtual switch. Test results will be outlined in details-of-LTR, preceded by the document-identifier and the scope and references).

This document is currently in draft form.

4.1.1. Document identifier

The document id will be used to uniquely identify versions of the LTR. The format for the document id will be: OPNFV_vswitchperf_LTR_rel_STATUS, the status is one of: DRAFT, REVIEWED, CORRECTED or FINAL. The document id for this version of the LTR is: OPNFV_vswitchperf_LTR_Brahmaputra_DRAFT.

4.1.2. Scope

The scope of this report is to detail the results of the tests that have been performed on the virtual switch. This report will also evaluate the results of these tests and, based on these evaluations, provide recommendations on the suitability of the virtual switch for use in a Telco NFV environment.

4.2. Details of the Level Test Report

This section provides a test-results-overview. Also included are the rationale and the conclusions.

4.2.1. Test ID: BACK2BACK

4.2.1.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 998842df061caf3f90adf756c78667262525a6e0

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

Below are test details:

  • Test ID: back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: p2p
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
4.2.1.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.1.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 5108
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:42:18
start_time 2018-04-24 14:30:56
stop_time 2018-04-24 15:13:14
4.2.1.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 22616
%usr 0.54
%system 1.05
%guest 0.00
%CPU 1.60
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 3364068
RSS 7657
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 22611
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5804
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.1.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 3382
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:42:18
start_time 2018-04-24 14:30:56
stop_time 2018-04-24 15:13:14
4.2.1.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 22616
%usr 0.54
%system 1.05
%guest 0.00
%CPU 1.60
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 3364068
RSS 7657
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 22611
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5804
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.1.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1344
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:42:18
start_time 2018-04-24 14:30:56
stop_time 2018-04-24 15:13:14
4.2.1.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 22616
%usr 0.54
%system 1.05
%guest 0.00
%CPU 1.60
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 3364068
RSS 7657
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 22611
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5804
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.1.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1096
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:42:18
start_time 2018-04-24 14:30:56
stop_time 2018-04-24 15:13:14
4.2.1.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 22616
%usr 0.54
%system 1.05
%guest 0.00
%CPU 1.60
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 3364068
RSS 7657
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 22611
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5804
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.1.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1116
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:42:18
start_time 2018-04-24 14:30:56
stop_time 2018-04-24 15:13:14
4.2.1.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 22616
%usr 0.54
%system 1.05
%guest 0.00
%CPU 1.60
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 3364068
RSS 7657
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 22611
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5804
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.7. Anomalies

No anomalies were detected during the course of this test.

4.2.1.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.2. Test ID: PHY2PHY_SCALABILITY

4.2.2.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 998842df061caf3f90adf756c78667262525a6e0

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

Below are test details:

  • Test ID: phy2phy_scalability
  • Description: LTD.Scalability.Flows.RFC2544.0PacketLoss
  • Deployment: p2p
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
4.2.2.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.2.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 988190.092
tx_rate_mbps Unknown
throughput_rx_mbps 505.953
tx_rate_percent 3.320
throughput_rx_percent 3.320
frame_loss_percent 0.000
min_latency_ns 4000.000
max_latency_ns 3083220.000
avg_latency_ns 44847.500
type rfc2544_throughput
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:41
start_time 2018-04-24 15:40:51
stop_time 2018-04-24 16:08:32
stream_count 8000
match_type L4
pre-installed_flows No
4.2.2.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 26700
%usr 0.55
%system 1.12
%guest 0.00
%CPU 1.67
CPU
minflt/s 0.86
majflt/s 0.00
VSZ 3364068
RSS 7701
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 26695
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44276
RSS 5804
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.2.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.2.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 952811.037
tx_rate_mbps Unknown
throughput_rx_mbps 975.679
tx_rate_percent 5.641
throughput_rx_percent 5.641
frame_loss_percent 0.000
min_latency_ns 4120.000
max_latency_ns 2267380.000
avg_latency_ns 35213.000
type rfc2544_throughput
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:41
start_time 2018-04-24 15:40:51
stop_time 2018-04-24 16:08:32
stream_count 8000
match_type L4
pre-installed_flows No
4.2.2.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 26700
%usr 0.55
%system 1.12
%guest 0.00
%CPU 1.67
CPU
minflt/s 0.86
majflt/s 0.00
VSZ 3364068
RSS 7701
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 26695
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44276
RSS 5804
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.2.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.2.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 846601.342
tx_rate_mbps Unknown
throughput_rx_mbps 3467.679
tx_rate_percent 18.016
throughput_rx_percent 18.016
frame_loss_percent 0.000
min_latency_ns 5140.000
max_latency_ns 1156060.000
avg_latency_ns 31445.500
type rfc2544_throughput
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:41
start_time 2018-04-24 15:40:51
stop_time 2018-04-24 16:08:32
stream_count 8000
match_type L4
pre-installed_flows No
4.2.2.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 26700
%usr 0.55
%system 1.12
%guest 0.00
%CPU 1.67
CPU
minflt/s 0.86
majflt/s 0.00
VSZ 3364068
RSS 7701
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 26695
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44276
RSS 5804
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.2.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.2.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 857395.182
tx_rate_mbps Unknown
throughput_rx_mbps 7023.781
tx_rate_percent 35.805
throughput_rx_percent 35.805
frame_loss_percent 0.000
min_latency_ns 5740.000
max_latency_ns 613440.000
avg_latency_ns 32801.500
type rfc2544_throughput
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:41
start_time 2018-04-24 15:40:51
stop_time 2018-04-24 16:08:32
stream_count 8000
match_type L4
pre-installed_flows No
4.2.2.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 26700
%usr 0.55
%system 1.12
%guest 0.00
%CPU 1.67
CPU
minflt/s 0.86
majflt/s 0.00
VSZ 3364068
RSS 7701
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 26695
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44276
RSS 5804
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.2.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.2.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 871160.461
tx_rate_mbps Unknown
throughput_rx_mbps 10579.373
tx_rate_percent 53.594
throughput_rx_percent 53.594
frame_loss_percent 0.000
min_latency_ns 8780.000
max_latency_ns 1063640.000
avg_latency_ns 58991.500
type rfc2544_throughput
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:41
start_time 2018-04-24 15:40:51
stop_time 2018-04-24 16:08:32
stream_count 8000
match_type L4
pre-installed_flows No
4.2.2.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 26700
%usr 0.55
%system 1.12
%guest 0.00
%CPU 1.67
CPU
minflt/s 0.86
majflt/s 0.00
VSZ 3364068
RSS 7701
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 26695
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44276
RSS 5804
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.2.7. Anomalies

No anomalies were detected during the course of this test.

4.2.2.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.3. Test ID: PVVP_TPUT

4.2.3.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvvp

  • vswitchperf: GIT tag: 998842df061caf3f90adf756c78667262525a6e0

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • VNF: QemuVirtioNet, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvvp_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvvp
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
4.2.3.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.3.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 260897.648
tx_rate_mbps Unknown
throughput_rx_mbps 133.580
tx_rate_percent 1.773
throughput_rx_percent 0.877
frame_loss_percent 50.568
min_latency_ns 31400.000
max_latency_ns 64345420.000
avg_latency_ns 4384490.500
type rfc2544_throughput
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:29:12
start_time 2018-04-24 17:20:02
stop_time 2018-04-24 17:49:15
guest_loopback_app testpmd testpmd
4.2.3.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 32893
%usr 0.56
%system 0.97
%guest 0.00
%CPU 1.52
CPU
minflt/s 0.13
majflt/s 0.00
VSZ 3364192
RSS 8096
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 32887
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5940
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 33200
%usr 0.09
%system 1.84
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2779749
RSS 65992
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.40
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 33129
%usr 0.02
%system 1.77
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2783524
RSS 66016
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
4.2.3.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.3.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 299569.034
tx_rate_mbps Unknown
throughput_rx_mbps 306.759
tx_rate_percent 1.773
throughput_rx_percent 1.773
frame_loss_percent 0.000
min_latency_ns 23340.000
max_latency_ns 628360.000
avg_latency_ns 69321.500
type rfc2544_throughput
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:29:12
start_time 2018-04-24 17:20:02
stop_time 2018-04-24 17:49:15
guest_loopback_app testpmd testpmd
4.2.3.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 32893
%usr 0.56
%system 0.97
%guest 0.00
%CPU 1.52
CPU
minflt/s 0.13
majflt/s 0.00
VSZ 3364192
RSS 8096
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 32887
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5940
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 33200
%usr 0.09
%system 1.84
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2779749
RSS 65992
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.40
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 33129
%usr 0.02
%system 1.77
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2783524
RSS 66016
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
4.2.3.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.3.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 192376.186
tx_rate_mbps Unknown
throughput_rx_mbps 787.973
tx_rate_percent 4.094
throughput_rx_percent 4.094
frame_loss_percent 0.000
min_latency_ns 29000.000
max_latency_ns 2473100.000
avg_latency_ns 84353.500
type rfc2544_throughput
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:29:12
start_time 2018-04-24 17:20:02
stop_time 2018-04-24 17:49:15
guest_loopback_app testpmd testpmd
4.2.3.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 32893
%usr 0.56
%system 0.97
%guest 0.00
%CPU 1.52
CPU
minflt/s 0.13
majflt/s 0.00
VSZ 3364192
RSS 8096
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 32887
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5940
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 33200
%usr 0.09
%system 1.84
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2779749
RSS 65992
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.40
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 33129
%usr 0.02
%system 1.77
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2783524
RSS 66016
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
4.2.3.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.3.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 79509.737
tx_rate_mbps Unknown
throughput_rx_mbps 651.344
tx_rate_percent 3.320
throughput_rx_percent 3.320
frame_loss_percent 0.000
min_latency_ns 24400.000
max_latency_ns 344900.000
avg_latency_ns 79506.500
type rfc2544_throughput
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:29:12
start_time 2018-04-24 17:20:02
stop_time 2018-04-24 17:49:15
guest_loopback_app testpmd testpmd
4.2.3.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 32893
%usr 0.56
%system 0.97
%guest 0.00
%CPU 1.52
CPU
minflt/s 0.13
majflt/s 0.00
VSZ 3364192
RSS 8096
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 32887
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5940
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 33200
%usr 0.09
%system 1.84
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2779749
RSS 65992
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.40
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 33129
%usr 0.02
%system 1.77
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2783524
RSS 66016
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
4.2.3.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.3.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 280270.818
tx_rate_mbps Unknown
throughput_rx_mbps 3403.609
tx_rate_percent 17.242
throughput_rx_percent 17.242
frame_loss_percent 0.000
min_latency_ns 31520.000
max_latency_ns 681820.000
avg_latency_ns 96479.500
type rfc2544_throughput
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:29:12
start_time 2018-04-24 17:20:02
stop_time 2018-04-24 17:49:15
guest_loopback_app testpmd testpmd
4.2.3.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 32893
%usr 0.56
%system 0.97
%guest 0.00
%CPU 1.52
CPU
minflt/s 0.13
majflt/s 0.00
VSZ 3364192
RSS 8096
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 32887
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5940
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 33200
%usr 0.09
%system 1.84
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2779749
RSS 65992
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.40
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 33129
%usr 0.02
%system 1.77
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2783524
RSS 66016
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
4.2.3.7. Anomalies

No anomalies were detected during the course of this test.

4.2.3.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.4. Test ID: PHY2PHY_TPUT_MOD_VLAN

4.2.4.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 998842df061caf3f90adf756c78667262525a6e0

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

Below are test details:

  • Test ID: phy2phy_tput_mod_vlan
  • Description: LTD.Throughput.RFC2544.PacketLossRatioFrameModification
  • Deployment: p2p
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
4.2.4.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.4.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 758000.832
tx_rate_mbps Unknown
throughput_rx_mbps 412.352
tx_rate_percent 2.547
throughput_rx_percent 2.668
frame_loss_percent 0.000
min_latency_ns 4040.000
max_latency_ns 273280.000
avg_latency_ns 30863.500
type rfc2544_throughput
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:35
start_time 2018-04-24 15:13:14
stop_time 2018-04-24 15:40:50
4.2.4.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 25004
%usr 0.55
%system 1.09
%guest 0.00
%CPU 1.64
CPU
minflt/s 0.16
majflt/s 0.00
VSZ 3364068
RSS 7698
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 24999
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5764
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.4.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.4.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 952811.395
tx_rate_mbps Unknown
throughput_rx_mbps 1006.169
tx_rate_percent 5.641
throughput_rx_percent 5.793
frame_loss_percent 0.000
min_latency_ns 4180.000
max_latency_ns 2728020.000
avg_latency_ns 37760.000
type rfc2544_throughput
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:35
start_time 2018-04-24 15:13:14
stop_time 2018-04-24 15:40:50
4.2.4.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 25004
%usr 0.55
%system 1.09
%guest 0.00
%CPU 1.64
CPU
minflt/s 0.16
majflt/s 0.00
VSZ 3364068
RSS 7698
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 24999
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5764
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.4.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.4.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 846599.988
tx_rate_mbps Unknown
throughput_rx_mbps 3494.765
tx_rate_percent 18.016
throughput_rx_percent 18.151
frame_loss_percent 0.000
min_latency_ns 5260.000
max_latency_ns 599380.000
avg_latency_ns 31272.500
type rfc2544_throughput
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:35
start_time 2018-04-24 15:13:14
stop_time 2018-04-24 15:40:50
4.2.4.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 25004
%usr 0.55
%system 1.09
%guest 0.00
%CPU 1.64
CPU
minflt/s 0.16
majflt/s 0.00
VSZ 3364068
RSS 7698
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 24999
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5764
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.4.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.4.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 857394.796
tx_rate_mbps Unknown
throughput_rx_mbps 7051.215
tx_rate_percent 35.805
throughput_rx_percent 35.942
frame_loss_percent 0.000
min_latency_ns 5800.000
max_latency_ns 850420.000
avg_latency_ns 36640.000
type rfc2544_throughput
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:35
start_time 2018-04-24 15:13:14
stop_time 2018-04-24 15:40:50
4.2.4.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 25004
%usr 0.55
%system 1.09
%guest 0.00
%CPU 1.64
CPU
minflt/s 0.16
majflt/s 0.00
VSZ 3364068
RSS 7698
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 24999
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5764
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.4.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.4.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 846017.971
tx_rate_mbps Unknown
throughput_rx_mbps 10301.115
tx_rate_percent 52.047
throughput_rx_percent 52.182
frame_loss_percent 0.000
min_latency_ns 8080.000
max_latency_ns 452080.000
avg_latency_ns 35906.500
type rfc2544_throughput
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:35
start_time 2018-04-24 15:13:14
stop_time 2018-04-24 15:40:50
4.2.4.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 25004
%usr 0.55
%system 1.09
%guest 0.00
%CPU 1.64
CPU
minflt/s 0.16
majflt/s 0.00
VSZ 3364068
RSS 7698
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 24999
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5764
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.4.7. Anomalies

No anomalies were detected during the course of this test.

4.2.4.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.5. Test ID: PVVP_BACK2BACK

4.2.5.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvvp

  • vswitchperf: GIT tag: 998842df061caf3f90adf756c78667262525a6e0

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • VNF: QemuVirtioNet, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvvp_back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: pvvp
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
4.2.5.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.5.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 425
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:55
start_time 2018-04-24 17:49:16
stop_time 2018-04-24 18:33:11
guest_loopback_app testpmd testpmd
4.2.5.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 34963
%usr 0.55
%system 1.02
%guest 0.00
%CPU 1.57
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 3364192
RSS 8023
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 34958
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5664
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 35275
%usr 0.03
%system 0.11
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2806140
RSS 65116
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 7.88
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 35196
%usr 0.02
%system 0.11
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2842120
RSS 61544
%MEM 0.09
kB_rd/s 0.00
kB_wr/s 0.02
kB_ccwr/s 0.00
4.2.5.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.5.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 966
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:55
start_time 2018-04-24 17:49:16
stop_time 2018-04-24 18:33:11
guest_loopback_app testpmd testpmd
4.2.5.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 34963
%usr 0.55
%system 1.02
%guest 0.00
%CPU 1.57
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 3364192
RSS 8023
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 34958
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5664
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 35275
%usr 0.03
%system 0.11
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2806140
RSS 65116
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 7.88
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 35196
%usr 0.02
%system 0.11
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2842120
RSS 61544
%MEM 0.09
kB_rd/s 0.00
kB_wr/s 0.02
kB_ccwr/s 0.00
4.2.5.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.5.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1075
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:55
start_time 2018-04-24 17:49:16
stop_time 2018-04-24 18:33:11
guest_loopback_app testpmd testpmd
4.2.5.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 34963
%usr 0.55
%system 1.02
%guest 0.00
%CPU 1.57
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 3364192
RSS 8023
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 34958
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5664
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 35275
%usr 0.03
%system 0.11
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2806140
RSS 65116
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 7.88
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 35196
%usr 0.02
%system 0.11
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2842120
RSS 61544
%MEM 0.09
kB_rd/s 0.00
kB_wr/s 0.02
kB_ccwr/s 0.00
4.2.5.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.5.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 548
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:55
start_time 2018-04-24 17:49:16
stop_time 2018-04-24 18:33:11
guest_loopback_app testpmd testpmd
4.2.5.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 34963
%usr 0.55
%system 1.02
%guest 0.00
%CPU 1.57
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 3364192
RSS 8023
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 34958
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5664
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 35275
%usr 0.03
%system 0.11
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2806140
RSS 65116
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 7.88
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 35196
%usr 0.02
%system 0.11
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2842120
RSS 61544
%MEM 0.09
kB_rd/s 0.00
kB_wr/s 0.02
kB_ccwr/s 0.00
4.2.5.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.5.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 744
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:55
start_time 2018-04-24 17:49:16
stop_time 2018-04-24 18:33:11
guest_loopback_app testpmd testpmd
4.2.5.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 34963
%usr 0.55
%system 1.02
%guest 0.00
%CPU 1.57
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 3364192
RSS 8023
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 34958
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5664
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 35275
%usr 0.03
%system 0.11
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2806140
RSS 65116
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 7.88
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 35196
%usr 0.02
%system 0.11
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2842120
RSS 61544
%MEM 0.09
kB_rd/s 0.00
kB_wr/s 0.02
kB_ccwr/s 0.00
4.2.5.7. Anomalies

No anomalies were detected during the course of this test.

4.2.5.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.6. Test ID: PVP_BACK2BACK

4.2.6.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: 998842df061caf3f90adf756c78667262525a6e0

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • VNF: QemuVirtioNet, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvp_back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: pvp
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
4.2.6.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.6.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 425
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:03
start_time 2018-04-24 16:36:59
stop_time 2018-04-24 17:20:02
guest_loopback_app testpmd
4.2.6.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 30272
%usr 0.54
%system 1.07
%guest 0.00
%CPU 1.61
CPU
minflt/s 0.10
majflt/s 0.00
VSZ 3364068
RSS 7600
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 30267
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5832
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 30460
%usr 0.03
%system 0.09
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2819504
RSS 62568
%MEM 0.09
kB_rd/s 0.00
kB_wr/s 8.61
kB_ccwr/s 0.00
4.2.6.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.6.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544_back2back
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:03
start_time 2018-04-24 16:36:59
stop_time 2018-04-24 17:20:02
guest_loopback_app testpmd
4.2.6.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 30272
%usr 0.54
%system 1.07
%guest 0.00
%CPU 1.61
CPU
minflt/s 0.10
majflt/s 0.00
VSZ 3364068
RSS 7600
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 30267
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5832
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 30460
%usr 0.03
%system 0.09
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2819504
RSS 62568
%MEM 0.09
kB_rd/s 0.00
kB_wr/s 8.61
kB_ccwr/s 0.00
4.2.6.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.6.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 537
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:03
start_time 2018-04-24 16:36:59
stop_time 2018-04-24 17:20:02
guest_loopback_app testpmd
4.2.6.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 30272
%usr 0.54
%system 1.07
%guest 0.00
%CPU 1.61
CPU
minflt/s 0.10
majflt/s 0.00
VSZ 3364068
RSS 7600
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 30267
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5832
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 30460
%usr 0.03
%system 0.09
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2819504
RSS 62568
%MEM 0.09
kB_rd/s 0.00
kB_wr/s 8.61
kB_ccwr/s 0.00
4.2.6.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.6.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 548
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:03
start_time 2018-04-24 16:36:59
stop_time 2018-04-24 17:20:02
guest_loopback_app testpmd
4.2.6.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 30272
%usr 0.54
%system 1.07
%guest 0.00
%CPU 1.61
CPU
minflt/s 0.10
majflt/s 0.00
VSZ 3364068
RSS 7600
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 30267
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5832
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 30460
%usr 0.03
%system 0.09
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2819504
RSS 62568
%MEM 0.09
kB_rd/s 0.00
kB_wr/s 8.61
kB_ccwr/s 0.00
4.2.6.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.6.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 372
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:03
start_time 2018-04-24 16:36:59
stop_time 2018-04-24 17:20:02
guest_loopback_app testpmd
4.2.6.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 30272
%usr 0.54
%system 1.07
%guest 0.00
%CPU 1.61
CPU
minflt/s 0.10
majflt/s 0.00
VSZ 3364068
RSS 7600
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 30267
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5832
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 30460
%usr 0.03
%system 0.09
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2819504
RSS 62568
%MEM 0.09
kB_rd/s 0.00
kB_wr/s 8.61
kB_ccwr/s 0.00
4.2.6.7. Anomalies

No anomalies were detected during the course of this test.

4.2.6.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.7. Test ID: PHY2PHY_TPUT

4.2.7.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 998842df061caf3f90adf756c78667262525a6e0

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

Below are test details:

  • Test ID: phy2phy_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: p2p
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
4.2.7.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.7.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 988190.952
tx_rate_mbps Unknown
throughput_rx_mbps 505.954
tx_rate_percent 3.320
throughput_rx_percent 3.320
frame_loss_percent 0.000
min_latency_ns 4000.000
max_latency_ns 2659580.000
avg_latency_ns 39277.500
type rfc2544_throughput
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:29:20
start_time 2018-04-24 14:01:35
stop_time 2018-04-24 14:30:55
4.2.7.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 20807
%usr 0.56
%system 1.05
%guest 0.00
%CPU 1.60
CPU
minflt/s 0.14
majflt/s 0.00
VSZ 3364068
RSS 7620
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 20791
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5952
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.7.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.7.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 952810.897
tx_rate_mbps Unknown
throughput_rx_mbps 975.678
tx_rate_percent 5.641
throughput_rx_percent 5.641
frame_loss_percent 0.000
min_latency_ns 4140.000
max_latency_ns 1240520.000
avg_latency_ns 31318.000
type rfc2544_throughput
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:29:20
start_time 2018-04-24 14:01:35
stop_time 2018-04-24 14:30:55
4.2.7.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 20807
%usr 0.56
%system 1.05
%guest 0.00
%CPU 1.60
CPU
minflt/s 0.14
majflt/s 0.00
VSZ 3364068
RSS 7620
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 20791
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5952
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.7.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.7.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 846599.945
tx_rate_mbps Unknown
throughput_rx_mbps 3467.673
tx_rate_percent 18.016
throughput_rx_percent 18.016
frame_loss_percent 0.000
min_latency_ns 5000.000
max_latency_ns 675180.000
avg_latency_ns 30221.500
type rfc2544_throughput
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:29:20
start_time 2018-04-24 14:01:35
stop_time 2018-04-24 14:30:55
4.2.7.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 20807
%usr 0.56
%system 1.05
%guest 0.00
%CPU 1.60
CPU
minflt/s 0.14
majflt/s 0.00
VSZ 3364068
RSS 7620
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 20791
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5952
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.7.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.7.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 875915.836
tx_rate_mbps Unknown
throughput_rx_mbps 7175.503
tx_rate_percent 36.578
throughput_rx_percent 36.578
frame_loss_percent 0.000
min_latency_ns 5760.000
max_latency_ns 707720.000
avg_latency_ns 38236.000
type rfc2544_throughput
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:29:20
start_time 2018-04-24 14:01:35
stop_time 2018-04-24 14:30:55
4.2.7.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 20807
%usr 0.56
%system 1.05
%guest 0.00
%CPU 1.60
CPU
minflt/s 0.14
majflt/s 0.00
VSZ 3364068
RSS 7620
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 20791
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5952
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.7.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.7.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 883734.268
tx_rate_mbps Unknown
throughput_rx_mbps 10732.069
tx_rate_percent 54.367
throughput_rx_percent 54.367
frame_loss_percent 0.000
min_latency_ns 8420.000
max_latency_ns 1068840.000
avg_latency_ns 58745.000
type rfc2544_throughput
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:29:20
start_time 2018-04-24 14:01:35
stop_time 2018-04-24 14:30:55
4.2.7.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 20807
%usr 0.56
%system 1.05
%guest 0.00
%CPU 1.60
CPU
minflt/s 0.14
majflt/s 0.00
VSZ 3364068
RSS 7620
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 20791
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5952
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.7.7. Anomalies

No anomalies were detected during the course of this test.

4.2.7.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.8. Test ID: PVP_TPUT

4.2.8.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: 998842df061caf3f90adf756c78667262525a6e0

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • VNF: QemuVirtioNet, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvp_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvp
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
4.2.8.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.8.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 311737.067
tx_rate_mbps Unknown
throughput_rx_mbps 159.609
tx_rate_percent 1.773
throughput_rx_percent 1.047
frame_loss_percent 40.935
min_latency_ns 11000.000
max_latency_ns 85259620.000
avg_latency_ns 3007614.000
type rfc2544_throughput
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:28:25
start_time 2018-04-24 16:08:33
stop_time 2018-04-24 16:36:58
guest_loopback_app testpmd
4.2.8.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 28369
%usr 0.54
%system 1.04
%guest 0.00
%CPU 1.59
CPU
minflt/s 0.19
majflt/s 0.00
VSZ 3364072
RSS 7695
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 28363
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5688
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 28556
%usr 0.06
%system 1.52
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2748572
RSS 66707
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.03
kB_ccwr/s 0.00
4.2.8.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.8.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 299567.928
tx_rate_mbps Unknown
throughput_rx_mbps 306.758
tx_rate_percent 1.773
throughput_rx_percent 1.773
frame_loss_percent 0.000
min_latency_ns 10960.000
max_latency_ns 1208820.000
avg_latency_ns 48662.500
type rfc2544_throughput
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:28:25
start_time 2018-04-24 16:08:33
stop_time 2018-04-24 16:36:58
guest_loopback_app testpmd
4.2.8.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 28369
%usr 0.54
%system 1.04
%guest 0.00
%CPU 1.59
CPU
minflt/s 0.19
majflt/s 0.00
VSZ 3364072
RSS 7695
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 28363
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5688
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 28556
%usr 0.06
%system 1.52
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2748572
RSS 66707
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.03
kB_ccwr/s 0.00
4.2.8.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.8.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 228721.643
tx_rate_mbps Unknown
throughput_rx_mbps 936.844
tx_rate_percent 4.867
throughput_rx_percent 4.867
frame_loss_percent 0.000
min_latency_ns 16060.000
max_latency_ns 470360.000
avg_latency_ns 67161.500
type rfc2544_throughput
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:28:25
start_time 2018-04-24 16:08:33
stop_time 2018-04-24 16:36:58
guest_loopback_app testpmd
4.2.8.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 28369
%usr 0.54
%system 1.04
%guest 0.00
%CPU 1.59
CPU
minflt/s 0.19
majflt/s 0.00
VSZ 3364072
RSS 7695
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 28363
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5688
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 28556
%usr 0.06
%system 1.52
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2748572
RSS 66707
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.03
kB_ccwr/s 0.00
4.2.8.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.8.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 264719.443
tx_rate_mbps Unknown
throughput_rx_mbps 2168.582
tx_rate_percent 11.055
throughput_rx_percent 11.055
frame_loss_percent 0.000
min_latency_ns 17060.000
max_latency_ns 875940.000
avg_latency_ns 69904.000
type rfc2544_throughput
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:28:25
start_time 2018-04-24 16:08:33
stop_time 2018-04-24 16:36:58
guest_loopback_app testpmd
4.2.8.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 28369
%usr 0.54
%system 1.04
%guest 0.00
%CPU 1.59
CPU
minflt/s 0.19
majflt/s 0.00
VSZ 3364072
RSS 7695
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 28363
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5688
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 28556
%usr 0.06
%system 1.52
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2748572
RSS 66707
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.03
kB_ccwr/s 0.00
4.2.8.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.8.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 267697.996
tx_rate_mbps Unknown
throughput_rx_mbps 3250.924
tx_rate_percent 16.469
throughput_rx_percent 16.469
frame_loss_percent 0.000
min_latency_ns 19780.000
max_latency_ns 797880.000
avg_latency_ns 74471.000
type rfc2544_throughput
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:28:25
start_time 2018-04-24 16:08:33
stop_time 2018-04-24 16:36:58
guest_loopback_app testpmd
4.2.8.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
UID 0
PID 28369
%usr 0.54
%system 1.04
%guest 0.00
%CPU 1.59
CPU
minflt/s 0.19
majflt/s 0.00
VSZ 3364072
RSS 7695
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
UID 0
PID 28363
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5688
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
UID 0
PID 28556
%usr 0.06
%system 1.52
%guest 100.00
%CPU 100.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2748572
RSS 66707
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.03
kB_ccwr/s 0.00
4.2.8.7. Anomalies

No anomalies were detected during the course of this test.

4.2.8.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.3. Rationale for decisions

The tests conducted do not have pass/fail/conditional-pass criteria. The test is simply conducted and the results are reported.

4.4. Conclusions and recommendations

The test results are stable. The vsperf CI jobs that were used to obtain the results can be found at https://artifacts.opnfv.org/logs/vswitchperf/intel-pod12/fraser/2018-04-24_17-25-03/vswitchperf_logs_2018-04-24_17-25-03.tar.gz.

4.5. General

4.5.1. Glossary

  • NFV - Network Function Virtualization
  • Mbps - 1,000,000bps

4.5.2. Document change procedures and history

Document ID Author Date Modified
OPNFV_vswitchperf_LTR_ver_1.0_Jan_15_CN_DRAFT Christopher Nolan 23/01/2015
OPNFV_vswitchperf_LTR_ver_1.1_Jan_15_CN_DRAFT Christopher Nolan 28/01/2015

5. Performance report for VppDpdkVhost

5.1. Introduction

The objective of the OPNFV project titled “Characterise vSwitch Performance for Telco NFV Use Cases”, is to evaluate a virtual switch to identify its suitability for a Telco Network Function Virtualization (NFV) environment. As well as this, the project aims to identify any gaps or bottlenecks in order to drive architectural changes to improve virtual switch performance and determinism. The purpose of this document is to summarize the results of the tests carried out on the virtual switch in the Network Function Virtualization Infrastructure (NFVI) and, from these results, provide evaluations and recommendations for the virtual switch. Test results will be outlined in details-of-LTR, preceded by the document-identifier and the scope and references).

This document is currently in draft form.

5.1.1. Document identifier

The document id will be used to uniquely identify versions of the LTR. The format for the document id will be: OPNFV_vswitchperf_LTR_rel_STATUS, the status is one of: DRAFT, REVIEWED, CORRECTED or FINAL. The document id for this version of the LTR is: OPNFV_vswitchperf_LTR_Brahmaputra_DRAFT.

5.1.2. Scope

The scope of this report is to detail the results of the tests that have been performed on the virtual switch. This report will also evaluate the results of these tests and, based on these evaluations, provide recommendations on the suitability of the virtual switch for use in a Telco NFV environment.

5.2. Details of the Level Test Report

This section provides a test-results-overview. Also included are the rationale and the conclusions.

5.2.1. Test ID: PVVP_BACK2BACK_VPP

5.2.1.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: clean

  • vswitchperf: GIT tag: 998842df061caf3f90adf756c78667262525a6e0

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: VppDpdkVhost, Version: v17.07-release, GIT tag: NA

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvvp_back2back_vpp
  • Description: VPP: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: clean
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
5.2.1.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

5.2.1.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544_back2back
packet_size 64
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:35:40
start_time 2018-04-24 20:08:35
stop_time 2018-04-24 20:44:16
5.2.1.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
UID 0
PID 44914
%usr 100.00
%system 0.14
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.08
majflt/s 0.00
VSZ 15476808
RSS 51399
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.1.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

5.2.1.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 483
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 128
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:35:40
start_time 2018-04-24 20:08:35
stop_time 2018-04-24 20:44:16
5.2.1.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
UID 0
PID 44914
%usr 100.00
%system 0.14
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.08
majflt/s 0.00
VSZ 15476808
RSS 51399
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.1.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

5.2.1.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 70488721
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:35:40
start_time 2018-04-24 20:08:35
stop_time 2018-04-24 20:44:16
5.2.1.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
UID 0
PID 44914
%usr 100.00
%system 0.14
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.08
majflt/s 0.00
VSZ 15476808
RSS 51399
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.1.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

5.2.1.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 35919540
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:35:40
start_time 2018-04-24 20:08:35
stop_time 2018-04-24 20:44:16
5.2.1.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
UID 0
PID 44914
%usr 100.00
%system 0.14
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.08
majflt/s 0.00
VSZ 15476808
RSS 51399
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.1.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

5.2.1.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 24382314
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:35:40
start_time 2018-04-24 20:08:35
stop_time 2018-04-24 20:44:16
5.2.1.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
UID 0
PID 44914
%usr 100.00
%system 0.14
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.08
majflt/s 0.00
VSZ 15476808
RSS 51399
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.1.7. Anomalies

No anomalies were detected during the course of this test.

5.2.1.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

5.2.2. Test ID: PHY2PHY_BACK2BACK_VPP

5.2.2.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: clean

  • vswitchperf: GIT tag: 998842df061caf3f90adf756c78667262525a6e0

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: VppDpdkVhost, Version: v17.07-release, GIT tag: NA

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: phy2phy_back2back_vpp
  • Description: VPP: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: clean
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
5.2.2.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

5.2.2.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 24267
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 64
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:33:53
start_time 2018-04-24 18:56:59
stop_time 2018-04-24 19:30:53
5.2.2.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
UID 0
PID 39659
%usr 100.00
%system 0.13
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 3937544
RSS 50747
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.2.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

5.2.2.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 247439
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 128
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:33:53
start_time 2018-04-24 18:56:59
stop_time 2018-04-24 19:30:53
5.2.2.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
UID 0
PID 39659
%usr 100.00
%system 0.13
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 3937544
RSS 50747
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.2.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

5.2.2.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 70488721
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:33:53
start_time 2018-04-24 18:56:59
stop_time 2018-04-24 19:30:53
5.2.2.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
UID 0
PID 39659
%usr 100.00
%system 0.13
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 3937544
RSS 50747
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.2.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

5.2.2.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 35919540
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:33:53
start_time 2018-04-24 18:56:59
stop_time 2018-04-24 19:30:53
5.2.2.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
UID 0
PID 39659
%usr 100.00
%system 0.13
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 3937544
RSS 50747
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.2.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

5.2.2.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 24382314
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:33:53
start_time 2018-04-24 18:56:59
stop_time 2018-04-24 19:30:53
5.2.2.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
UID 0
PID 39659
%usr 100.00
%system 0.13
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 3937544
RSS 50747
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.2.7. Anomalies

No anomalies were detected during the course of this test.

5.2.2.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

5.2.3. Test ID: PVVP_TPUT_VPP

5.2.3.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: clean

  • vswitchperf: GIT tag: 998842df061caf3f90adf756c78667262525a6e0

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: VppDpdkVhost, Version: v17.07-release, GIT tag: NA

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvvp_tput_vpp
  • Description: VPP: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: clean
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
5.2.3.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

5.2.3.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4210844.031
tx_rate_mbps Unknown
throughput_rx_mbps 2155.952
tx_rate_percent 14.148
throughput_rx_percent 14.148
frame_loss_percent 0.000
min_latency_ns 6860.000
max_latency_ns 80120.000
avg_latency_ns 13237.000
type rfc2544_throughput
packet_size 64
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:27:44
start_time 2018-04-24 19:40:50
stop_time 2018-04-24 20:08:34
5.2.3.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
UID 0
PID 42995
%usr 100.00
%system 0.14
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.14
majflt/s 0.00
VSZ 15168478
RSS 51679
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.3.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

5.2.3.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4741609.590
tx_rate_mbps Unknown
throughput_rx_mbps 4855.408
tx_rate_percent 28.070
throughput_rx_percent 28.070
frame_loss_percent 0.000
min_latency_ns 7760.000
max_latency_ns 75140.000
avg_latency_ns 16993.000
type rfc2544_throughput
packet_size 128
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:27:44
start_time 2018-04-24 19:40:50
stop_time 2018-04-24 20:08:34
5.2.3.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
UID 0
PID 42995
%usr 100.00
%system 0.14
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.14
majflt/s 0.00
VSZ 15168478
RSS 51679
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.3.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

5.2.3.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4335790.077
tx_rate_mbps Unknown
throughput_rx_mbps 17759.396
tx_rate_percent 92.266
throughput_rx_percent 92.266
frame_loss_percent 0.000
min_latency_ns 11720.000
max_latency_ns 99440.000
avg_latency_ns 39198.500
type rfc2544_throughput
packet_size 512
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:27:44
start_time 2018-04-24 19:40:50
stop_time 2018-04-24 20:08:34
5.2.3.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
UID 0
PID 42995
%usr 100.00
%system 0.14
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.14
majflt/s 0.00
VSZ 15168478
RSS 51679
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.3.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

5.2.3.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394563.736
tx_rate_mbps Unknown
throughput_rx_mbps 19616.266
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 10500.000
max_latency_ns 112460.000
avg_latency_ns 90581.500
type rfc2544_throughput
packet_size 1024
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:27:44
start_time 2018-04-24 19:40:50
stop_time 2018-04-24 20:08:34
5.2.3.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
UID 0
PID 42995
%usr 100.00
%system 0.14
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.14
majflt/s 0.00
VSZ 15168478
RSS 51679
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.3.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

5.2.3.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625444.698
tx_rate_mbps Unknown
throughput_rx_mbps 19739.400
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 12880.000
max_latency_ns 106780.000
avg_latency_ns 82100.000
type rfc2544_throughput
packet_size 1518
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:27:44
start_time 2018-04-24 19:40:50
stop_time 2018-04-24 20:08:34
5.2.3.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
UID 0
PID 42995
%usr 100.00
%system 0.14
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.14
majflt/s 0.00
VSZ 15168478
RSS 51679
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.3.7. Anomalies

No anomalies were detected during the course of this test.

5.2.3.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

5.2.4. Test ID: PHY2PHY_TPUT_VPP

5.2.4.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: clean

  • vswitchperf: GIT tag: 998842df061caf3f90adf756c78667262525a6e0

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: VppDpdkVhost, Version: v17.07-release, GIT tag: NA

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: phy2phy_tput_vpp
  • Description: VPP: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: clean
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
5.2.4.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

5.2.4.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 22856313.652
tx_rate_mbps Unknown
throughput_rx_mbps 11702.433
tx_rate_percent 76.797
throughput_rx_percent 76.797
frame_loss_percent 0.000
min_latency_ns 5220.000
max_latency_ns 53680.000
avg_latency_ns 24597.500
type rfc2544_throughput
packet_size 64
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:23:36
start_time 2018-04-24 18:33:21
stop_time 2018-04-24 18:56:58
5.2.4.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
UID 0
PID 38118
%usr 100.00
%system 0.13
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 3937544
RSS 50954
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.4.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

5.2.4.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 16630607.917
tx_rate_mbps Unknown
throughput_rx_mbps 17029.743
tx_rate_percent 98.453
throughput_rx_percent 98.453
frame_loss_percent 0.000
min_latency_ns 5480.000
max_latency_ns 37980.000
avg_latency_ns 13143.500
type rfc2544_throughput
packet_size 128
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:23:36
start_time 2018-04-24 18:33:21
stop_time 2018-04-24 18:56:58
5.2.4.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
UID 0
PID 38118
%usr 100.00
%system 0.13
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 3937544
RSS 50954
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.4.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

5.2.4.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699206.419
tx_rate_mbps Unknown
throughput_rx_mbps 19247.949
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 5480.000
max_latency_ns 32420.000
avg_latency_ns 21259.000
type rfc2544_throughput
packet_size 512
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:23:36
start_time 2018-04-24 18:33:21
stop_time 2018-04-24 18:56:58
5.2.4.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
UID 0
PID 38118
%usr 100.00
%system 0.13
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 3937544
RSS 50954
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.4.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

5.2.4.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394613.598
tx_rate_mbps Unknown
throughput_rx_mbps 19616.675
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 6460.000
max_latency_ns 34940.000
avg_latency_ns 23922.500
type rfc2544_throughput
packet_size 1024
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:23:36
start_time 2018-04-24 18:33:21
stop_time 2018-04-24 18:56:58
5.2.4.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
UID 0
PID 38118
%usr 100.00
%system 0.13
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 3937544
RSS 50954
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.4.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

5.2.4.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625475.321
tx_rate_mbps Unknown
throughput_rx_mbps 19739.772
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 6700.000
max_latency_ns 29820.000
avg_latency_ns 19437.000
type rfc2544_throughput
packet_size 1518
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:23:36
start_time 2018-04-24 18:33:21
stop_time 2018-04-24 18:56:58
5.2.4.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
UID 0
PID 38118
%usr 100.00
%system 0.13
%guest 0.00
%CPU 100.00
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 3937544
RSS 50954
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.4.7. Anomalies

No anomalies were detected during the course of this test.

5.2.4.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

5.3. Rationale for decisions

The tests conducted do not have pass/fail/conditional-pass criteria. The test is simply conducted and the results are reported.

5.4. Conclusions and recommendations

The test results are stable. The vsperf CI jobs that were used to obtain the results can be found at https://artifacts.opnfv.org/logs/vswitchperf/intel-pod12/fraser/2018-04-24_17-25-03/vswitchperf_logs_2018-04-24_17-25-03.tar.gz.

5.5. General

5.5.1. Glossary

  • NFV - Network Function Virtualization
  • Mbps - 1,000,000bps

5.5.2. Document change procedures and history

Document ID Author Date Modified
OPNFV_vswitchperf_LTR_ver_1.0_Jan_15_CN_DRAFT Christopher Nolan 23/01/2015
OPNFV_vswitchperf_LTR_ver_1.1_Jan_15_CN_DRAFT Christopher Nolan 28/01/2015

6. Performance report for none

6.1. Introduction

The objective of the OPNFV project titled “Characterise vSwitch Performance for Telco NFV Use Cases”, is to evaluate a virtual switch to identify its suitability for a Telco Network Function Virtualization (NFV) environment. As well as this, the project aims to identify any gaps or bottlenecks in order to drive architectural changes to improve virtual switch performance and determinism. The purpose of this document is to summarize the results of the tests carried out on the virtual switch in the Network Function Virtualization Infrastructure (NFVI) and, from these results, provide evaluations and recommendations for the virtual switch. Test results will be outlined in details-of-LTR, preceded by the document-identifier and the scope and references).

This document is currently in draft form.

6.1.1. Document identifier

The document id will be used to uniquely identify versions of the LTR. The format for the document id will be: OPNFV_vswitchperf_LTR_rel_STATUS, the status is one of: DRAFT, REVIEWED, CORRECTED or FINAL. The document id for this version of the LTR is: OPNFV_vswitchperf_LTR_Brahmaputra_DRAFT.

6.1.2. Scope

The scope of this report is to detail the results of the tests that have been performed on the virtual switch. This report will also evaluate the results of these tests and, based on these evaluations, provide recommendations on the suitability of the virtual switch for use in a Telco NFV environment.

6.2. Details of the Level Test Report

This section provides a test-results-overview. Also included are the rationale and the conclusions.

6.2.1. Test ID: PVP_TPUT

6.2.1.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: 998842df061caf3f90adf756c78667262525a6e0

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: none, Version: None, GIT tag: None

  • VNF: QemuPciPassthrough, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvp_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvp
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
6.2.1.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

6.2.1.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 0
tx_rate_mbps Unknown
throughput_rx_mbps 0
tx_rate_percent 1.773
throughput_rx_percent 0.000
frame_loss_percent 100.000
min_latency_ns 0
max_latency_ns 0
avg_latency_ns 0
type rfc2544_throughput
packet_size 64
vswitch none
traffic_type udp
test_execution_time 00:29:10
start_time 2018-04-24 20:44:23
stop_time 2018-04-24 21:13:34
guest_loopback_app testpmd
6.2.1.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
UID 0
PID 47505
%usr 0.03
%system 0.05
%guest 0.03
%CPU 0.11
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2823312
RSS 76279
%MEM 0.12
kB_rd/s 0.00
kB_wr/s 11.65
kB_ccwr/s 0.00
6.2.1.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

6.2.1.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 0
tx_rate_mbps Unknown
throughput_rx_mbps 0
tx_rate_percent 1.773
throughput_rx_percent 0.000
frame_loss_percent 100.000
min_latency_ns 0
max_latency_ns 0
avg_latency_ns 0
type rfc2544_throughput
packet_size 128
vswitch none
traffic_type udp
test_execution_time 00:29:10
start_time 2018-04-24 20:44:23
stop_time 2018-04-24 21:13:34
guest_loopback_app testpmd
6.2.1.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
UID 0
PID 47505
%usr 0.03
%system 0.05
%guest 0.03
%CPU 0.11
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2823312
RSS 76279
%MEM 0.12
kB_rd/s 0.00
kB_wr/s 11.65
kB_ccwr/s 0.00
6.2.1.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

6.2.1.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 0
tx_rate_mbps Unknown
throughput_rx_mbps 0
tx_rate_percent 1.773
throughput_rx_percent 0.000
frame_loss_percent 100.000
min_latency_ns 0
max_latency_ns 0
avg_latency_ns 0
type rfc2544_throughput
packet_size 512
vswitch none
traffic_type udp
test_execution_time 00:29:10
start_time 2018-04-24 20:44:23
stop_time 2018-04-24 21:13:34
guest_loopback_app testpmd
6.2.1.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
UID 0
PID 47505
%usr 0.03
%system 0.05
%guest 0.03
%CPU 0.11
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2823312
RSS 76279
%MEM 0.12
kB_rd/s 0.00
kB_wr/s 11.65
kB_ccwr/s 0.00
6.2.1.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

6.2.1.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 0
tx_rate_mbps Unknown
throughput_rx_mbps 0
tx_rate_percent 1.773
throughput_rx_percent 0.000
frame_loss_percent 100.000
min_latency_ns 0
max_latency_ns 0
avg_latency_ns 0
type rfc2544_throughput
packet_size 1024
vswitch none
traffic_type udp
test_execution_time 00:29:10
start_time 2018-04-24 20:44:23
stop_time 2018-04-24 21:13:34
guest_loopback_app testpmd
6.2.1.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
UID 0
PID 47505
%usr 0.03
%system 0.05
%guest 0.03
%CPU 0.11
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2823312
RSS 76279
%MEM 0.12
kB_rd/s 0.00
kB_wr/s 11.65
kB_ccwr/s 0.00
6.2.1.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

6.2.1.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 0
tx_rate_mbps Unknown
throughput_rx_mbps 0
tx_rate_percent 1.773
throughput_rx_percent 0.000
frame_loss_percent 100.000
min_latency_ns 0
max_latency_ns 0
avg_latency_ns 0
type rfc2544_throughput
packet_size 1518
vswitch none
traffic_type udp
test_execution_time 00:29:10
start_time 2018-04-24 20:44:23
stop_time 2018-04-24 21:13:34
guest_loopback_app testpmd
6.2.1.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
UID 0
PID 47505
%usr 0.03
%system 0.05
%guest 0.03
%CPU 0.11
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2823312
RSS 76279
%MEM 0.12
kB_rd/s 0.00
kB_wr/s 11.65
kB_ccwr/s 0.00
6.2.1.7. Anomalies

No anomalies were detected during the course of this test.

6.2.1.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

6.3. Rationale for decisions

The tests conducted do not have pass/fail/conditional-pass criteria. The test is simply conducted and the results are reported.

6.4. Conclusions and recommendations

The test results are stable. The vsperf CI jobs that were used to obtain the results can be found at https://artifacts.opnfv.org/logs/vswitchperf/intel-pod12/fraser/2018-04-24_17-25-03/vswitchperf_logs_2018-04-24_17-25-03.tar.gz.

6.5. General

6.5.1. Glossary

  • NFV - Network Function Virtualization
  • Mbps - 1,000,000bps

6.5.2. Document change procedures and history

Document ID Author Date Modified
OPNFV_vswitchperf_LTR_ver_1.0_Jan_15_CN_DRAFT Christopher Nolan 23/01/2015
OPNFV_vswitchperf_LTR_ver_1.1_Jan_15_CN_DRAFT Christopher Nolan 28/01/2015