VSPERF Results

1. VSPERF Test Scenarios

Predefined Tests suitable for automated execution with CI:

Test Definition
phy2phy_tput PacketLossRatio for Phy2Phy
back2back BackToBackFrames for Phy2Phy
phy2phy_tput_mod_vlan PacketLossRatioFrameModification for Phy2Phy
phy2phy_cont Phy2Phy blast vswitch at x% TX rate and measure throughput
pvp_cont PVP blast vswitch at x% TX rate and measure throughput
pvvp_cont PVVP blast vswitch at x% TX rate and measure throughput
phy2phy_scalability Scalability0PacketLoss for Phy2Phy
pvp_tput PacketLossRatio for PVP
pvp_back2back BackToBackFrames for PVP
pvvp_tput PacketLossRatio for PVVP
pvvp_back2back BackToBackFrames for PVVP
phy2phy_cpu_load CPU0PacketLoss for Phy2Phy
phy2phy_mem_load Same as CPU0PacketLoss but using a memory intensive app

Deployment topologies:

  • Phy2Phy: Physical port -> vSwitch -> Physical port.
  • PVP: Physical port -> vSwitch -> VNF -> vSwitch -> Physical port.
  • PVVP: Physical port -> vSwitch -> VNF -> vSwitch -> VNF -> vSwitch -> Physical port.

Loopback applications in the Guest:

Supported traffic generators:

  • Spirent Testcenter
  • Ixia: IxOS and IxNet.
  • Xena
  • MoonGen
  • Dummy
  • T-Rex

2. OPNFV Test Results

VSPERF CI jobs are run daily and sample results can be found at https://wiki.opnfv.org/display/vsperf/Vsperf+Results

Testcase names shown in the dashboard are combination of orignal testcase name from VSPERF framework and indication of used vswitch.

Example:

Testcase phy2phy_tput is executed for three vSwitch types: OvsDpdkVhost, OvsVanilla and VppDpdkVhost. In this case, following testcase names will be used in the dashboard: phy2phy_tput_ovsdpdkvhost, phy2phy_tput_ovsvanilla and phy2phy_tput_vppdpdkvhost.

In case of RFC2544 Throughput test, the recorded metric is FPS (frames per second) without packet loss. For RFC2544 Back2Back test, the recorded metric is back-to-back value (number of frames) without packet loss.

The loopback application in the VNF used for PVP and PVVP scenarios was DPDK testpmd.

Guest interface types are vhost-user for OvsDpdkVhost and VppDpdkVhost and virtio-net for OvsVanilla.

3. Performance report for OvsDpdkVhost

3.1. Introduction

The objective of the OPNFV project titled “Characterise vSwitch Performance for Telco NFV Use Cases”, is to evaluate a virtual switch to identify its suitability for a Telco Network Function Virtualization (NFV) environment. As well as this, the project aims to identify any gaps or bottlenecks in order to drive architectural changes to improve virtual switch performance and determinism. The purpose of this document is to summarize the results of the tests carried out on the virtual switch in the Network Function Virtualization Infrastructure (NFVI) and, from these results, provide evaluations and recommendations for the virtual switch. Test results will be outlined in details-of-LTR, preceded by the document-identifier and the scope and references).

This document is currently in draft form.

3.1.1. Document identifier

The document id will be used to uniquely identify versions of the LTR. The format for the document id will be: OPNFV_vswitchperf_LTR_rel_STATUS, the status is one of: DRAFT, REVIEWED, CORRECTED or FINAL. The document id for this version of the LTR is: OPNFV_vswitchperf_LTR_Brahmaputra_DRAFT.

3.1.2. Scope

The scope of this report is to detail the results of the tests that have been performed on the virtual switch. This report will also evaluate the results of these tests and, based on these evaluations, provide recommendations on the suitability of the virtual switch for use in a Telco NFV environment.

3.2. Details of the Level Test Report

This section provides a test-results-overview. Also included are the rationale and the conclusions.

3.2.1. Test ID: BACK2BACK

3.2.1.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 665dcb2b00632384762f2cf9754b2e2cdce6a69d

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: p2p
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
3.2.1.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.1.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 25970
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 64
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:34:01
start_time 2018-09-22 09:23:34
stop_time 2018-09-22 09:57:36
3.2.1.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.02
%system 0.02
%guest 0.00
%CPU 200.08
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056816
RSS 43087
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50704
RSS 6668
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.1.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 253378378
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 128
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:34:01
start_time 2018-09-22 09:23:34
stop_time 2018-09-22 09:57:36
3.2.1.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.02
%system 0.02
%guest 0.00
%CPU 200.08
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056816
RSS 43087
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50704
RSS 6668
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.1.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 70488721
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:34:01
start_time 2018-09-22 09:23:34
stop_time 2018-09-22 09:57:36
3.2.1.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.02
%system 0.02
%guest 0.00
%CPU 200.08
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056816
RSS 43087
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50704
RSS 6668
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.1.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 35919540
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:34:01
start_time 2018-09-22 09:23:34
stop_time 2018-09-22 09:57:36
3.2.1.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.02
%system 0.02
%guest 0.00
%CPU 200.08
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056816
RSS 43087
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50704
RSS 6668
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.1.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 24382314
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:34:01
start_time 2018-09-22 09:23:34
stop_time 2018-09-22 09:57:36
3.2.1.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.02
%system 0.02
%guest 0.00
%CPU 200.08
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056816
RSS 43087
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50704
RSS 6668
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.7. Anomalies

No anomalies were detected during the course of this test.

3.2.1.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.2. Test ID: PHY2PHY_SCALABILITY

3.2.2.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 665dcb2b00632384762f2cf9754b2e2cdce6a69d

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: phy2phy_scalability
  • Description: LTD.Scalability.Flows.RFC2544.0PacketLoss
  • Deployment: p2p
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
3.2.2.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.2.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 15720371.085
tx_rate_mbps Unknown
throughput_rx_mbps 8048.830
tx_rate_percent 52.820
throughput_rx_percent 52.820
frame_loss_percent 0.000
min_latency_ns 4860.000
max_latency_ns 55920.000
avg_latency_ns 23550.500
type rfc2544_throughput
packet_size 64
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:20:50
start_time 2018-09-22 10:15:56
stop_time 2018-09-22 10:36:47
stream_count 8000
match_type L4
pre-installed_flows No
3.2.2.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.02
%system 0.02
%guest 0.00
%CPU 200.05
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 5056816
RSS 43063
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6856
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.2.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.2.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 15716122.587
tx_rate_mbps Unknown
throughput_rx_mbps 16093.310
tx_rate_percent 93.039
throughput_rx_percent 93.039
frame_loss_percent 0.000
min_latency_ns 5940.000
max_latency_ns 54520.000
avg_latency_ns 26433.000
type rfc2544_throughput
packet_size 128
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:20:50
start_time 2018-09-22 10:15:56
stop_time 2018-09-22 10:36:47
stream_count 8000
match_type L4
pre-installed_flows No
3.2.2.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.02
%system 0.02
%guest 0.00
%CPU 200.05
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 5056816
RSS 43063
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6856
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.2.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.2.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699200.858
tx_rate_mbps Unknown
throughput_rx_mbps 19247.927
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 12560.000
max_latency_ns 47760.000
avg_latency_ns 32784.000
type rfc2544_throughput
packet_size 512
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:20:50
start_time 2018-09-22 10:15:56
stop_time 2018-09-22 10:36:47
stream_count 8000
match_type L4
pre-installed_flows No
3.2.2.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.02
%system 0.02
%guest 0.00
%CPU 200.05
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 5056816
RSS 43063
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6856
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.2.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.2.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394615.610
tx_rate_mbps Unknown
throughput_rx_mbps 19616.691
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 15420.000
max_latency_ns 42560.000
avg_latency_ns 31075.500
type rfc2544_throughput
packet_size 1024
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:20:50
start_time 2018-09-22 10:15:56
stop_time 2018-09-22 10:36:47
stream_count 8000
match_type L4
pre-installed_flows No
3.2.2.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.02
%system 0.02
%guest 0.00
%CPU 200.05
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 5056816
RSS 43063
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6856
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.2.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.2.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625475.137
tx_rate_mbps Unknown
throughput_rx_mbps 19739.770
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 13160.000
max_latency_ns 36640.000
avg_latency_ns 26180.500
type rfc2544_throughput
packet_size 1518
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:20:50
start_time 2018-09-22 10:15:56
stop_time 2018-09-22 10:36:47
stream_count 8000
match_type L4
pre-installed_flows No
3.2.2.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.02
%system 0.02
%guest 0.00
%CPU 200.05
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 5056816
RSS 43063
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6856
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.2.7. Anomalies

No anomalies were detected during the course of this test.

3.2.2.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.3. Test ID: PHY2PHY_TPUT_MOD_VLAN

3.2.3.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 665dcb2b00632384762f2cf9754b2e2cdce6a69d

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: phy2phy_tput_mod_vlan
  • Description: LTD.Throughput.RFC2544.PacketLossRatioFrameModification
  • Deployment: p2p
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
3.2.3.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.3.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 22165588.999
tx_rate_mbps Unknown
throughput_rx_mbps 11348.782
tx_rate_percent 74.477
throughput_rx_percent 74.477
frame_loss_percent 0.000
min_latency_ns 5060.000
max_latency_ns 212780.000
avg_latency_ns 28119.000
type rfc2544_throughput
packet_size 64
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:18:19
start_time 2018-09-22 09:57:36
stop_time 2018-09-22 10:15:55
3.2.3.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.01
%system 0.02
%guest 0.00
%CPU 200.09
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 5056808
RSS 42734
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6780
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.3.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.3.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 16891660.650
tx_rate_mbps Unknown
throughput_rx_mbps 17297.061
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 14000.000
max_latency_ns 56680.000
avg_latency_ns 45070.000
type rfc2544_throughput
packet_size 128
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:18:19
start_time 2018-09-22 09:57:36
stop_time 2018-09-22 10:15:55
3.2.3.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.01
%system 0.02
%guest 0.00
%CPU 200.09
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 5056808
RSS 42734
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6780
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.3.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.3.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699206.920
tx_rate_mbps Unknown
throughput_rx_mbps 19247.952
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 12800.000
max_latency_ns 39580.000
avg_latency_ns 29003.000
type rfc2544_throughput
packet_size 512
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:18:19
start_time 2018-09-22 09:57:36
stop_time 2018-09-22 10:15:55
3.2.3.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.01
%system 0.02
%guest 0.00
%CPU 200.09
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 5056808
RSS 42734
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6780
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.3.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.3.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394616.065
tx_rate_mbps Unknown
throughput_rx_mbps 19616.695
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 16400.000
max_latency_ns 42720.000
avg_latency_ns 31688.000
type rfc2544_throughput
packet_size 1024
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:18:19
start_time 2018-09-22 09:57:36
stop_time 2018-09-22 10:15:55
3.2.3.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.01
%system 0.02
%guest 0.00
%CPU 200.09
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 5056808
RSS 42734
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6780
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.3.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.3.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625475.017
tx_rate_mbps Unknown
throughput_rx_mbps 19739.769
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 14700.000
max_latency_ns 39260.000
avg_latency_ns 28482.000
type rfc2544_throughput
packet_size 1518
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:18:19
start_time 2018-09-22 09:57:36
stop_time 2018-09-22 10:15:55
3.2.3.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.01
%system 0.02
%guest 0.00
%CPU 200.09
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 5056808
RSS 42734
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6780
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.3.7. Anomalies

No anomalies were detected during the course of this test.

3.2.3.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.4. Test ID: PVP_BACK2BACK

3.2.4.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: 665dcb2b00632384762f2cf9754b2e2cdce6a69d

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

  • VNF: QemuDpdkVhostUser, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-16.04_trex_20180209.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvp_back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: pvp
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
3.2.4.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.4.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 2128
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 64
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:47:25
start_time 2018-09-22 11:00:04
stop_time 2018-09-22 11:47:30
guest_loopback_app testpmd
3.2.4.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.00
%system 0.02
%guest 0.00
%CPU 400.13
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 11422000
RSS 43208
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50704
RSS 6720
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.05
%system 0.05
%guest 200.01
%CPU 200.12
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2850247
RSS 68379
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 7.67
kB_ccwr/s 0.00
3.2.4.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.4.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1449
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 128
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:47:25
start_time 2018-09-22 11:00:04
stop_time 2018-09-22 11:47:30
guest_loopback_app testpmd
3.2.4.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.00
%system 0.02
%guest 0.00
%CPU 400.13
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 11422000
RSS 43208
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50704
RSS 6720
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.05
%system 0.05
%guest 200.01
%CPU 200.12
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2850247
RSS 68379
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 7.67
kB_ccwr/s 0.00
3.2.4.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.4.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 70488717
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:47:25
start_time 2018-09-22 11:00:04
stop_time 2018-09-22 11:47:30
guest_loopback_app testpmd
3.2.4.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.00
%system 0.02
%guest 0.00
%CPU 400.13
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 11422000
RSS 43208
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50704
RSS 6720
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.05
%system 0.05
%guest 200.01
%CPU 200.12
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2850247
RSS 68379
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 7.67
kB_ccwr/s 0.00
3.2.4.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.4.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 35919540
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:47:25
start_time 2018-09-22 11:00:04
stop_time 2018-09-22 11:47:30
guest_loopback_app testpmd
3.2.4.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.00
%system 0.02
%guest 0.00
%CPU 400.13
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 11422000
RSS 43208
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50704
RSS 6720
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.05
%system 0.05
%guest 200.01
%CPU 200.12
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2850247
RSS 68379
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 7.67
kB_ccwr/s 0.00
3.2.4.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.4.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 24382314
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:47:25
start_time 2018-09-22 11:00:04
stop_time 2018-09-22 11:47:30
guest_loopback_app testpmd
3.2.4.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.00
%system 0.02
%guest 0.00
%CPU 400.13
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 11422000
RSS 43208
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50704
RSS 6720
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.05
%system 0.05
%guest 200.01
%CPU 200.12
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2850247
RSS 68379
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 7.67
kB_ccwr/s 0.00
3.2.4.7. Anomalies

No anomalies were detected during the course of this test.

3.2.4.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.5. Test ID: PHY2PHY_TPUT

3.2.5.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 665dcb2b00632384762f2cf9754b2e2cdce6a69d

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: phy2phy_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: p2p
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
3.2.5.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.5.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 23316599.894
tx_rate_mbps Unknown
throughput_rx_mbps 11938.099
tx_rate_percent 78.344
throughput_rx_percent 78.344
frame_loss_percent 0.000
min_latency_ns 4640.000
max_latency_ns 22960.000
avg_latency_ns 11840.000
type rfc2544_throughput
packet_size 64
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:16:35
start_time 2018-09-22 09:06:59
stop_time 2018-09-22 09:23:34
3.2.5.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.02
%system 0.02
%guest 0.00
%CPU 200.05
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 5056816
RSS 43058
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6268
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.5.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.5.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 16891680.750
tx_rate_mbps Unknown
throughput_rx_mbps 17297.081
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 13100.000
max_latency_ns 56500.000
avg_latency_ns 40489.000
type rfc2544_throughput
packet_size 128
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:16:35
start_time 2018-09-22 09:06:59
stop_time 2018-09-22 09:23:34
3.2.5.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.02
%system 0.02
%guest 0.00
%CPU 200.05
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 5056816
RSS 43058
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6268
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.5.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.5.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699206.247
tx_rate_mbps Unknown
throughput_rx_mbps 19247.949
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 12960.000
max_latency_ns 40100.000
avg_latency_ns 29674.000
type rfc2544_throughput
packet_size 512
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:16:35
start_time 2018-09-22 09:06:59
stop_time 2018-09-22 09:23:34
3.2.5.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.02
%system 0.02
%guest 0.00
%CPU 200.05
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 5056816
RSS 43058
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6268
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.5.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.5.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394616.607
tx_rate_mbps Unknown
throughput_rx_mbps 19616.699
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 13140.000
max_latency_ns 38620.000
avg_latency_ns 27653.500
type rfc2544_throughput
packet_size 1024
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:16:35
start_time 2018-09-22 09:06:59
stop_time 2018-09-22 09:23:34
3.2.5.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.02
%system 0.02
%guest 0.00
%CPU 200.05
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 5056816
RSS 43058
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6268
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.5.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.5.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625475.429
tx_rate_mbps Unknown
throughput_rx_mbps 19739.774
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 13240.000
max_latency_ns 35960.000
avg_latency_ns 25744.000
type rfc2544_throughput
packet_size 1518
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:16:35
start_time 2018-09-22 09:06:59
stop_time 2018-09-22 09:23:34
3.2.5.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.02
%system 0.02
%guest 0.00
%CPU 200.05
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 5056816
RSS 43058
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6268
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.5.7. Anomalies

No anomalies were detected during the course of this test.

3.2.5.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.6. Test ID: PVP_TPUT

3.2.6.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: 665dcb2b00632384762f2cf9754b2e2cdce6a69d

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

  • VNF: QemuDpdkVhostUser, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-16.04_trex_20180209.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvp_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvp
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
3.2.6.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.6.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 3980658.339
tx_rate_mbps Unknown
throughput_rx_mbps 2038.097
tx_rate_percent 13.375
throughput_rx_percent 13.375
frame_loss_percent 0.000
min_latency_ns 4780.000
max_latency_ns 42240.000
avg_latency_ns 5510.500
type rfc2544_throughput
packet_size 64
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:23:16
start_time 2018-09-22 10:36:47
stop_time 2018-09-22 11:00:03
guest_loopback_app testpmd
3.2.6.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.00
%system 0.02
%guest 0.00
%CPU 400.14
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 11421996
RSS 43104
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6668
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.05
%system 0.04
%guest 200.01
%CPU 200.09
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2823496
RSS 68090
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 16.33
kB_ccwr/s 0.00
3.2.6.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.6.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 8530444.731
tx_rate_mbps Unknown
throughput_rx_mbps 8735.175
tx_rate_percent 50.500
throughput_rx_percent 50.500
frame_loss_percent 0.000
min_latency_ns 7440.000
max_latency_ns 42860.000
avg_latency_ns 9541.000
type rfc2544_throughput
packet_size 128
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:23:16
start_time 2018-09-22 10:36:47
stop_time 2018-09-22 11:00:03
guest_loopback_app testpmd
3.2.6.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.00
%system 0.02
%guest 0.00
%CPU 400.14
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 11421996
RSS 43104
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6668
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.05
%system 0.04
%guest 200.01
%CPU 200.09
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2823496
RSS 68090
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 16.33
kB_ccwr/s 0.00
3.2.6.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.6.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699167.040
tx_rate_mbps Unknown
throughput_rx_mbps 19247.788
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 25260.000
max_latency_ns 78960.000
avg_latency_ns 67116.000
type rfc2544_throughput
packet_size 512
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:23:16
start_time 2018-09-22 10:36:47
stop_time 2018-09-22 11:00:03
guest_loopback_app testpmd
3.2.6.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.00
%system 0.02
%guest 0.00
%CPU 400.14
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 11421996
RSS 43104
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6668
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.05
%system 0.04
%guest 200.01
%CPU 200.09
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2823496
RSS 68090
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 16.33
kB_ccwr/s 0.00
3.2.6.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.6.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2098301.096
tx_rate_mbps Unknown
throughput_rx_mbps 17189.283
tx_rate_percent 87.625
throughput_rx_percent 87.625
frame_loss_percent 0.000
min_latency_ns 6960.000
max_latency_ns 42260.000
avg_latency_ns 8341.500
type rfc2544_throughput
packet_size 1024
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:23:16
start_time 2018-09-22 10:36:47
stop_time 2018-09-22 11:00:03
guest_loopback_app testpmd
3.2.6.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.00
%system 0.02
%guest 0.00
%CPU 400.14
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 11421996
RSS 43104
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6668
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.05
%system 0.04
%guest 200.01
%CPU 200.09
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2823496
RSS 68090
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 16.33
kB_ccwr/s 0.00
3.2.6.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.6.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625470.190
tx_rate_mbps Unknown
throughput_rx_mbps 19739.710
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 24880.000
max_latency_ns 59100.000
avg_latency_ns 47745.500
type rfc2544_throughput
packet_size 1518
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:23:16
start_time 2018-09-22 10:36:47
stop_time 2018-09-22 11:00:03
guest_loopback_app testpmd
3.2.6.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.00
%system 0.02
%guest 0.00
%CPU 400.14
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 11421996
RSS 43104
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50708
RSS 6668
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.05
%system 0.04
%guest 200.01
%CPU 200.09
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2823496
RSS 68090
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 16.33
kB_ccwr/s 0.00
3.2.6.7. Anomalies

No anomalies were detected during the course of this test.

3.2.6.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.3. Rationale for decisions

The tests conducted do not have pass/fail/conditional-pass criteria. The test is simply conducted and the results are reported.

3.4. Conclusions and recommendations

The test results are stable. The vsperf CI jobs that were used to obtain the results can be found at https://artifacts.opnfv.org/logs/vswitchperf/intel-pod12/2018-09-22_16-06-51/vswitchperf_logs_2018-09-22_16-06-51.tar.gz.

3.5. General

3.5.1. Glossary

  • NFV - Network Function Virtualization
  • Mbps - 1,000,000bps

3.5.2. Document change procedures and history

Document ID Author Date Modified
OPNFV_vswitchperf_LTR_ver_1.0_Jan_15_CN_DRAFT Christopher Nolan 23/01/2015
OPNFV_vswitchperf_LTR_ver_1.1_Jan_15_CN_DRAFT Christopher Nolan 28/01/2015

4. Performance report for OvsVanilla

4.1. Introduction

The objective of the OPNFV project titled “Characterise vSwitch Performance for Telco NFV Use Cases”, is to evaluate a virtual switch to identify its suitability for a Telco Network Function Virtualization (NFV) environment. As well as this, the project aims to identify any gaps or bottlenecks in order to drive architectural changes to improve virtual switch performance and determinism. The purpose of this document is to summarize the results of the tests carried out on the virtual switch in the Network Function Virtualization Infrastructure (NFVI) and, from these results, provide evaluations and recommendations for the virtual switch. Test results will be outlined in details-of-LTR, preceded by the document-identifier and the scope and references).

This document is currently in draft form.

4.1.1. Document identifier

The document id will be used to uniquely identify versions of the LTR. The format for the document id will be: OPNFV_vswitchperf_LTR_rel_STATUS, the status is one of: DRAFT, REVIEWED, CORRECTED or FINAL. The document id for this version of the LTR is: OPNFV_vswitchperf_LTR_Brahmaputra_DRAFT.

4.1.2. Scope

The scope of this report is to detail the results of the tests that have been performed on the virtual switch. This report will also evaluate the results of these tests and, based on these evaluations, provide recommendations on the suitability of the virtual switch for use in a Telco NFV environment.

4.2. Details of the Level Test Report

This section provides a test-results-overview. Also included are the rationale and the conclusions.

4.2.1. Test ID: BACK2BACK

4.2.1.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 665dcb2b00632384762f2cf9754b2e2cdce6a69d

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

Below are test details:

  • Test ID: back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: p2p
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
4.2.1.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.1.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 5108
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:42:19
start_time 2018-09-22 12:33:04
stop_time 2018-09-22 13:15:23
4.2.1.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.20
%system 1.23
%guest 0.00
%CPU 1.42
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 3364112
RSS 7660
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44316
RSS 5920
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.1.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 3382
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:42:19
start_time 2018-09-22 12:33:04
stop_time 2018-09-22 13:15:23
4.2.1.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.20
%system 1.23
%guest 0.00
%CPU 1.42
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 3364112
RSS 7660
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44316
RSS 5920
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.1.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1344
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:42:19
start_time 2018-09-22 12:33:04
stop_time 2018-09-22 13:15:23
4.2.1.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.20
%system 1.23
%guest 0.00
%CPU 1.42
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 3364112
RSS 7660
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44316
RSS 5920
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.1.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1370
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:42:19
start_time 2018-09-22 12:33:04
stop_time 2018-09-22 13:15:23
4.2.1.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.20
%system 1.23
%guest 0.00
%CPU 1.42
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 3364112
RSS 7660
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44316
RSS 5920
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.1.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1488
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:42:19
start_time 2018-09-22 12:33:04
stop_time 2018-09-22 13:15:23
4.2.1.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.20
%system 1.23
%guest 0.00
%CPU 1.42
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 3364112
RSS 7660
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44316
RSS 5920
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.7. Anomalies

No anomalies were detected during the course of this test.

4.2.1.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.2. Test ID: PHY2PHY_SCALABILITY

4.2.2.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 665dcb2b00632384762f2cf9754b2e2cdce6a69d

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

Below are test details:

  • Test ID: phy2phy_scalability
  • Description: LTD.Scalability.Flows.RFC2544.0PacketLoss
  • Deployment: p2p
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
4.2.2.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.2.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 758000.268
tx_rate_mbps Unknown
throughput_rx_mbps 388.096
tx_rate_percent 2.547
throughput_rx_percent 2.547
frame_loss_percent 0.000
min_latency_ns 4060.000
max_latency_ns 456700.000
avg_latency_ns 31646.000
type rfc2544_throughput
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:44
start_time 2018-09-22 13:43:08
stop_time 2018-09-22 14:10:53
stream_count 8000
match_type L4
pre-installed_flows No
4.2.2.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.21
%system 1.33
%guest 0.00
%CPU 1.49
CPU
minflt/s 1.58
majflt/s 0.00
VSZ 3364108
RSS 13192
%MEM 0.02
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5792
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.2.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.2.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 952811.135
tx_rate_mbps Unknown
throughput_rx_mbps 975.679
tx_rate_percent 5.641
throughput_rx_percent 5.641
frame_loss_percent 0.000
min_latency_ns 4200.000
max_latency_ns 3834620.000
avg_latency_ns 58114.500
type rfc2544_throughput
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:44
start_time 2018-09-22 13:43:08
stop_time 2018-09-22 14:10:53
stream_count 8000
match_type L4
pre-installed_flows No
4.2.2.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.21
%system 1.33
%guest 0.00
%CPU 1.49
CPU
minflt/s 1.58
majflt/s 0.00
VSZ 3364108
RSS 13192
%MEM 0.02
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5792
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.2.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.2.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 810255.975
tx_rate_mbps Unknown
throughput_rx_mbps 3318.808
tx_rate_percent 17.242
throughput_rx_percent 17.242
frame_loss_percent 0.000
min_latency_ns 5040.000
max_latency_ns 1215300.000
avg_latency_ns 32393.500
type rfc2544_throughput
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:44
start_time 2018-09-22 13:43:08
stop_time 2018-09-22 14:10:53
stream_count 8000
match_type L4
pre-installed_flows No
4.2.2.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.21
%system 1.33
%guest 0.00
%CPU 1.49
CPU
minflt/s 1.58
majflt/s 0.00
VSZ 3364108
RSS 13192
%MEM 0.02
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5792
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.2.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.2.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 838873.801
tx_rate_mbps Unknown
throughput_rx_mbps 6872.054
tx_rate_percent 35.031
throughput_rx_percent 35.031
frame_loss_percent 0.000
min_latency_ns 5800.000
max_latency_ns 901220.000
avg_latency_ns 41439.000
type rfc2544_throughput
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:44
start_time 2018-09-22 13:43:08
stop_time 2018-09-22 14:10:53
stream_count 8000
match_type L4
pre-installed_flows No
4.2.2.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.21
%system 1.33
%guest 0.00
%CPU 1.49
CPU
minflt/s 1.58
majflt/s 0.00
VSZ 3364108
RSS 13192
%MEM 0.02
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5792
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.2.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.2.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 846016.257
tx_rate_mbps Unknown
throughput_rx_mbps 10274.021
tx_rate_percent 52.047
throughput_rx_percent 52.047
frame_loss_percent 0.000
min_latency_ns 8400.000
max_latency_ns 900360.000
avg_latency_ns 55806.500
type rfc2544_throughput
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:44
start_time 2018-09-22 13:43:08
stop_time 2018-09-22 14:10:53
stream_count 8000
match_type L4
pre-installed_flows No
4.2.2.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.21
%system 1.33
%guest 0.00
%CPU 1.49
CPU
minflt/s 1.58
majflt/s 0.00
VSZ 3364108
RSS 13192
%MEM 0.02
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5792
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.2.7. Anomalies

No anomalies were detected during the course of this test.

4.2.2.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.3. Test ID: PVVP_TPUT

4.2.3.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvvp

  • vswitchperf: GIT tag: 665dcb2b00632384762f2cf9754b2e2cdce6a69d

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • VNF: QemuVirtioNet, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-16.04_trex_20180209.qcow2
    • /home/jenkins/vloop-vnf-ubuntu-16.04_trex_20180209.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvvp_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvvp
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
4.2.3.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.3.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 241801.973
tx_rate_mbps Unknown
throughput_rx_mbps 123.803
tx_rate_percent 1.773
throughput_rx_percent 0.812
frame_loss_percent 54.185
min_latency_ns 32160.000
max_latency_ns 78449900.000
avg_latency_ns 4969299.500
type rfc2544_throughput
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:29:41
start_time 2018-09-22 15:23:08
stop_time 2018-09-22 15:52:49
guest_loopback_app testpmd testpmd
4.2.3.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.17
%system 1.12
%guest 0.00
%CPU 1.27
CPU
minflt/s 0.15
majflt/s 0.00
VSZ 3364236
RSS 8066
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 6008
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.06
%system 1.69
%guest 198.40
%CPU 200.14
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 3062173
RSS 69523
%MEM 0.11
kB_rd/s 0.00
kB_wr/s 13.01
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.11
%system 1.72
%guest 198.45
%CPU 200.27
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2790740
RSS 67576
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.3.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.3.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 299550.425
tx_rate_mbps Unknown
throughput_rx_mbps 306.740
tx_rate_percent 1.773
throughput_rx_percent 1.773
frame_loss_percent 0.006
min_latency_ns 19800.000
max_latency_ns 5281600.000
avg_latency_ns 63797.000
type rfc2544_throughput
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:29:41
start_time 2018-09-22 15:23:08
stop_time 2018-09-22 15:52:49
guest_loopback_app testpmd testpmd
4.2.3.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.17
%system 1.12
%guest 0.00
%CPU 1.27
CPU
minflt/s 0.15
majflt/s 0.00
VSZ 3364236
RSS 8066
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 6008
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.06
%system 1.69
%guest 198.40
%CPU 200.14
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 3062173
RSS 69523
%MEM 0.11
kB_rd/s 0.00
kB_wr/s 13.01
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.11
%system 1.72
%guest 198.45
%CPU 200.27
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2790740
RSS 67576
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.3.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.3.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 156030.087
tx_rate_mbps Unknown
throughput_rx_mbps 639.099
tx_rate_percent 3.320
throughput_rx_percent 3.320
frame_loss_percent 0.000
min_latency_ns 24120.000
max_latency_ns 351420.000
avg_latency_ns 77433.500
type rfc2544_throughput
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:29:41
start_time 2018-09-22 15:23:08
stop_time 2018-09-22 15:52:49
guest_loopback_app testpmd testpmd
4.2.3.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.17
%system 1.12
%guest 0.00
%CPU 1.27
CPU
minflt/s 0.15
majflt/s 0.00
VSZ 3364236
RSS 8066
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 6008
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.06
%system 1.69
%guest 198.40
%CPU 200.14
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 3062173
RSS 69523
%MEM 0.11
kB_rd/s 0.00
kB_wr/s 13.01
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.11
%system 1.72
%guest 198.45
%CPU 200.27
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2790740
RSS 67576
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.3.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.3.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 283242.122
tx_rate_mbps Unknown
throughput_rx_mbps 2320.319
tx_rate_percent 11.828
throughput_rx_percent 11.828
frame_loss_percent 0.000
min_latency_ns 29640.000
max_latency_ns 1076820.000
avg_latency_ns 104182.500
type rfc2544_throughput
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:29:41
start_time 2018-09-22 15:23:08
stop_time 2018-09-22 15:52:49
guest_loopback_app testpmd testpmd
4.2.3.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.17
%system 1.12
%guest 0.00
%CPU 1.27
CPU
minflt/s 0.15
majflt/s 0.00
VSZ 3364236
RSS 8066
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 6008
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.06
%system 1.69
%guest 198.40
%CPU 200.14
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 3062173
RSS 69523
%MEM 0.11
kB_rd/s 0.00
kB_wr/s 13.01
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.11
%system 1.72
%guest 198.45
%CPU 200.27
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2790740
RSS 67576
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.3.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.3.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 267698.546
tx_rate_mbps Unknown
throughput_rx_mbps 3250.931
tx_rate_percent 16.469
throughput_rx_percent 16.469
frame_loss_percent 0.000
min_latency_ns 32980.000
max_latency_ns 965420.000
avg_latency_ns 97382.500
type rfc2544_throughput
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:29:41
start_time 2018-09-22 15:23:08
stop_time 2018-09-22 15:52:49
guest_loopback_app testpmd testpmd
4.2.3.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.17
%system 1.12
%guest 0.00
%CPU 1.27
CPU
minflt/s 0.15
majflt/s 0.00
VSZ 3364236
RSS 8066
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 6008
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.06
%system 1.69
%guest 198.40
%CPU 200.14
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 3062173
RSS 69523
%MEM 0.11
kB_rd/s 0.00
kB_wr/s 13.01
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.11
%system 1.72
%guest 198.45
%CPU 200.27
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2790740
RSS 67576
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.3.7. Anomalies

No anomalies were detected during the course of this test.

4.2.3.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.4. Test ID: PHY2PHY_TPUT_MOD_VLAN

4.2.4.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 665dcb2b00632384762f2cf9754b2e2cdce6a69d

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

Below are test details:

  • Test ID: phy2phy_tput_mod_vlan
  • Description: LTD.Throughput.RFC2544.PacketLossRatioFrameModification
  • Deployment: p2p
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
4.2.4.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.4.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 758000.363
tx_rate_mbps Unknown
throughput_rx_mbps 388.096
tx_rate_percent 2.547
throughput_rx_percent 2.547
frame_loss_percent 0.000
min_latency_ns 4040.000
max_latency_ns 334880.000
avg_latency_ns 31628.500
type rfc2544_throughput
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:44
start_time 2018-09-22 13:15:23
stop_time 2018-09-22 13:43:08
4.2.4.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.22
%system 1.25
%guest 0.00
%CPU 1.48
CPU
minflt/s 0.15
majflt/s 0.00
VSZ 3364112
RSS 7802
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5880
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.4.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.4.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 822161.346
tx_rate_mbps Unknown
throughput_rx_mbps 841.893
tx_rate_percent 4.867
throughput_rx_percent 4.867
frame_loss_percent 0.000
min_latency_ns 4200.000
max_latency_ns 1294480.000
avg_latency_ns 32792.000
type rfc2544_throughput
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:44
start_time 2018-09-22 13:15:23
stop_time 2018-09-22 13:43:08
4.2.4.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.22
%system 1.25
%guest 0.00
%CPU 1.48
CPU
minflt/s 0.15
majflt/s 0.00
VSZ 3364112
RSS 7802
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5880
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.4.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.4.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 846599.887
tx_rate_mbps Unknown
throughput_rx_mbps 3467.673
tx_rate_percent 18.016
throughput_rx_percent 18.016
frame_loss_percent 0.000
min_latency_ns 5280.000
max_latency_ns 2416700.000
avg_latency_ns 46011.000
type rfc2544_throughput
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:44
start_time 2018-09-22 13:15:23
stop_time 2018-09-22 13:43:08
4.2.4.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.22
%system 1.25
%guest 0.00
%CPU 1.48
CPU
minflt/s 0.15
majflt/s 0.00
VSZ 3364112
RSS 7802
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5880
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.4.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.4.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 857394.339
tx_rate_mbps Unknown
throughput_rx_mbps 7023.774
tx_rate_percent 35.805
throughput_rx_percent 35.805
frame_loss_percent 0.000
min_latency_ns 5780.000
max_latency_ns 1303660.000
avg_latency_ns 54177.000
type rfc2544_throughput
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:44
start_time 2018-09-22 13:15:23
stop_time 2018-09-22 13:43:08
4.2.4.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.22
%system 1.25
%guest 0.00
%CPU 1.48
CPU
minflt/s 0.15
majflt/s 0.00
VSZ 3364112
RSS 7802
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5880
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.4.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.4.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 846018.141
tx_rate_mbps Unknown
throughput_rx_mbps 10274.044
tx_rate_percent 52.047
throughput_rx_percent 52.047
frame_loss_percent 0.000
min_latency_ns 8420.000
max_latency_ns 552760.000
avg_latency_ns 42599.500
type rfc2544_throughput
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:44
start_time 2018-09-22 13:15:23
stop_time 2018-09-22 13:43:08
4.2.4.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.22
%system 1.25
%guest 0.00
%CPU 1.48
CPU
minflt/s 0.15
majflt/s 0.00
VSZ 3364112
RSS 7802
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5880
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.4.7. Anomalies

No anomalies were detected during the course of this test.

4.2.4.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.5. Test ID: PVVP_BACK2BACK

4.2.5.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvvp

  • vswitchperf: GIT tag: 665dcb2b00632384762f2cf9754b2e2cdce6a69d

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • VNF: QemuVirtioNet, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-16.04_trex_20180209.qcow2
    • /home/jenkins/vloop-vnf-ubuntu-16.04_trex_20180209.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvvp_back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: pvvp
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
4.2.5.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.5.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 425
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:44:23
start_time 2018-09-22 15:52:49
stop_time 2018-09-22 16:37:13
guest_loopback_app testpmd testpmd
4.2.5.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.23
%system 1.20
%guest 0.00
%CPU 1.39
CPU
minflt/s 0.14
majflt/s 0.00
VSZ 3364236
RSS 8088
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5936
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.04
%system 0.14
%guest 199.90
%CPU 200.08
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2794593
RSS 67619
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.34
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.05
%system 0.16
%guest 199.90
%CPU 200.11
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 3470952
RSS 68009
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.5.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.5.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 483
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:44:23
start_time 2018-09-22 15:52:49
stop_time 2018-09-22 16:37:13
guest_loopback_app testpmd testpmd
4.2.5.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.23
%system 1.20
%guest 0.00
%CPU 1.39
CPU
minflt/s 0.14
majflt/s 0.00
VSZ 3364236
RSS 8088
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5936
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.04
%system 0.14
%guest 199.90
%CPU 200.08
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2794593
RSS 67619
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.34
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.05
%system 0.16
%guest 199.90
%CPU 200.11
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 3470952
RSS 68009
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.5.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.5.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 806
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:44:23
start_time 2018-09-22 15:52:49
stop_time 2018-09-22 16:37:13
guest_loopback_app testpmd testpmd
4.2.5.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.23
%system 1.20
%guest 0.00
%CPU 1.39
CPU
minflt/s 0.14
majflt/s 0.00
VSZ 3364236
RSS 8088
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5936
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.04
%system 0.14
%guest 199.90
%CPU 200.08
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2794593
RSS 67619
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.34
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.05
%system 0.16
%guest 199.90
%CPU 200.11
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 3470952
RSS 68009
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.5.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.5.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 548
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:44:23
start_time 2018-09-22 15:52:49
stop_time 2018-09-22 16:37:13
guest_loopback_app testpmd testpmd
4.2.5.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.23
%system 1.20
%guest 0.00
%CPU 1.39
CPU
minflt/s 0.14
majflt/s 0.00
VSZ 3364236
RSS 8088
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5936
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.04
%system 0.14
%guest 199.90
%CPU 200.08
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2794593
RSS 67619
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.34
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.05
%system 0.16
%guest 199.90
%CPU 200.11
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 3470952
RSS 68009
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.5.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.5.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 744
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:44:23
start_time 2018-09-22 15:52:49
stop_time 2018-09-22 16:37:13
guest_loopback_app testpmd testpmd
4.2.5.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.23
%system 1.20
%guest 0.00
%CPU 1.39
CPU
minflt/s 0.14
majflt/s 0.00
VSZ 3364236
RSS 8088
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5936
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.04
%system 0.14
%guest 199.90
%CPU 200.08
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2794593
RSS 67619
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.34
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.05
%system 0.16
%guest 199.90
%CPU 200.11
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 3470952
RSS 68009
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.5.7. Anomalies

No anomalies were detected during the course of this test.

4.2.5.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.6. Test ID: PVP_BACK2BACK

4.2.6.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: 665dcb2b00632384762f2cf9754b2e2cdce6a69d

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • VNF: QemuVirtioNet, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-16.04_trex_20180209.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvp_back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: pvp
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
4.2.6.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.6.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1702
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:24
start_time 2018-09-22 14:39:43
stop_time 2018-09-22 15:23:07
guest_loopback_app testpmd
4.2.6.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.20
%system 1.23
%guest 0.00
%CPU 1.40
CPU
minflt/s 0.14
majflt/s 0.00
VSZ 3364104
RSS 7753
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5868
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.06
%system 0.14
%guest 199.94
%CPU 200.15
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2791640
RSS 67347
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.46
kB_ccwr/s 0.00
4.2.6.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.6.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544_back2back
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:24
start_time 2018-09-22 14:39:43
stop_time 2018-09-22 15:23:07
guest_loopback_app testpmd
4.2.6.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.20
%system 1.23
%guest 0.00
%CPU 1.40
CPU
minflt/s 0.14
majflt/s 0.00
VSZ 3364104
RSS 7753
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5868
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.06
%system 0.14
%guest 199.94
%CPU 200.15
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2791640
RSS 67347
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.46
kB_ccwr/s 0.00
4.2.6.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.6.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 268
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:24
start_time 2018-09-22 14:39:43
stop_time 2018-09-22 15:23:07
guest_loopback_app testpmd
4.2.6.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.20
%system 1.23
%guest 0.00
%CPU 1.40
CPU
minflt/s 0.14
majflt/s 0.00
VSZ 3364104
RSS 7753
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5868
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.06
%system 0.14
%guest 199.94
%CPU 200.15
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2791640
RSS 67347
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.46
kB_ccwr/s 0.00
4.2.6.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.6.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 548
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:24
start_time 2018-09-22 14:39:43
stop_time 2018-09-22 15:23:07
guest_loopback_app testpmd
4.2.6.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.20
%system 1.23
%guest 0.00
%CPU 1.40
CPU
minflt/s 0.14
majflt/s 0.00
VSZ 3364104
RSS 7753
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5868
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.06
%system 0.14
%guest 199.94
%CPU 200.15
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2791640
RSS 67347
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.46
kB_ccwr/s 0.00
4.2.6.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.6.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 744
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:24
start_time 2018-09-22 14:39:43
stop_time 2018-09-22 15:23:07
guest_loopback_app testpmd
4.2.6.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.20
%system 1.23
%guest 0.00
%CPU 1.40
CPU
minflt/s 0.14
majflt/s 0.00
VSZ 3364104
RSS 7753
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5868
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.06
%system 0.14
%guest 199.94
%CPU 200.15
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2791640
RSS 67347
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.46
kB_ccwr/s 0.00
4.2.6.7. Anomalies

No anomalies were detected during the course of this test.

4.2.6.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.7. Test ID: PHY2PHY_TPUT

4.2.7.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 665dcb2b00632384762f2cf9754b2e2cdce6a69d

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

Below are test details:

  • Test ID: phy2phy_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: p2p
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
4.2.7.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.7.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 758000.215
tx_rate_mbps Unknown
throughput_rx_mbps 388.096
tx_rate_percent 2.547
throughput_rx_percent 2.547
frame_loss_percent 0.000
min_latency_ns 4040.000
max_latency_ns 382780.000
avg_latency_ns 31729.000
type rfc2544_throughput
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:36
start_time 2018-09-22 12:05:27
stop_time 2018-09-22 12:33:03
4.2.7.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.19
%system 1.22
%guest 0.00
%CPU 1.45
CPU
minflt/s 0.16
majflt/s 0.00
VSZ 3364104
RSS 7781
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5832
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.7.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.7.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 952810.856
tx_rate_mbps Unknown
throughput_rx_mbps 975.678
tx_rate_percent 5.641
throughput_rx_percent 5.641
frame_loss_percent 0.000
min_latency_ns 4200.000
max_latency_ns 2689620.000
avg_latency_ns 40774.500
type rfc2544_throughput
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:36
start_time 2018-09-22 12:05:27
stop_time 2018-09-22 12:33:03
4.2.7.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.19
%system 1.22
%guest 0.00
%CPU 1.45
CPU
minflt/s 0.16
majflt/s 0.00
VSZ 3364104
RSS 7781
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5832
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.7.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.7.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 810256.017
tx_rate_mbps Unknown
throughput_rx_mbps 3318.809
tx_rate_percent 17.242
throughput_rx_percent 17.242
frame_loss_percent 0.000
min_latency_ns 5040.000
max_latency_ns 826640.000
avg_latency_ns 31617.500
type rfc2544_throughput
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:36
start_time 2018-09-22 12:05:27
stop_time 2018-09-22 12:33:03
4.2.7.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.19
%system 1.22
%guest 0.00
%CPU 1.45
CPU
minflt/s 0.16
majflt/s 0.00
VSZ 3364104
RSS 7781
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5832
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.7.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.7.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 857394.693
tx_rate_mbps Unknown
throughput_rx_mbps 7023.777
tx_rate_percent 35.805
throughput_rx_percent 35.805
frame_loss_percent 0.000
min_latency_ns 5800.000
max_latency_ns 1643280.000
avg_latency_ns 63802.500
type rfc2544_throughput
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:36
start_time 2018-09-22 12:05:27
stop_time 2018-09-22 12:33:03
4.2.7.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.19
%system 1.22
%guest 0.00
%CPU 1.45
CPU
minflt/s 0.16
majflt/s 0.00
VSZ 3364104
RSS 7781
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5832
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.7.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.7.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 846017.829
tx_rate_mbps Unknown
throughput_rx_mbps 10274.041
tx_rate_percent 52.047
throughput_rx_percent 52.047
frame_loss_percent 0.000
min_latency_ns 9060.000
max_latency_ns 869560.000
avg_latency_ns 48627.500
type rfc2544_throughput
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:36
start_time 2018-09-22 12:05:27
stop_time 2018-09-22 12:33:03
4.2.7.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.19
%system 1.22
%guest 0.00
%CPU 1.45
CPU
minflt/s 0.16
majflt/s 0.00
VSZ 3364104
RSS 7781
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5832
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.7.7. Anomalies

No anomalies were detected during the course of this test.

4.2.7.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.8. Test ID: PVP_TPUT

4.2.8.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: 665dcb2b00632384762f2cf9754b2e2cdce6a69d

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • VNF: QemuVirtioNet, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-16.04_trex_20180209.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvp_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvp
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
4.2.8.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.8.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 227808.608
tx_rate_mbps Unknown
throughput_rx_mbps 116.638
tx_rate_percent 1.773
throughput_rx_percent 0.765
frame_loss_percent 56.837
min_latency_ns 16180.000
max_latency_ns 80341320.000
avg_latency_ns 5446205.000
type rfc2544_throughput
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:28:49
start_time 2018-09-22 14:10:53
stop_time 2018-09-22 14:39:42
guest_loopback_app testpmd
4.2.8.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.19
%system 1.20
%guest 0.00
%CPU 1.39
CPU
minflt/s 0.23
majflt/s 0.00
VSZ 3364104
RSS 7767
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5900
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 1.04
%guest 198.99
%CPU 200.07
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2868554
RSS 66638
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 12.57
kB_ccwr/s 0.00
4.2.8.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.8.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 299567.743
tx_rate_mbps Unknown
throughput_rx_mbps 306.757
tx_rate_percent 1.773
throughput_rx_percent 1.773
frame_loss_percent 0.000
min_latency_ns 11260.000
max_latency_ns 1210300.000
avg_latency_ns 48095.000
type rfc2544_throughput
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:28:49
start_time 2018-09-22 14:10:53
stop_time 2018-09-22 14:39:42
guest_loopback_app testpmd
4.2.8.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.19
%system 1.20
%guest 0.00
%CPU 1.39
CPU
minflt/s 0.23
majflt/s 0.00
VSZ 3364104
RSS 7767
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5900
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 1.04
%guest 198.99
%CPU 200.07
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2868554
RSS 66638
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 12.57
kB_ccwr/s 0.00
4.2.8.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.8.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 265067.373
tx_rate_mbps Unknown
throughput_rx_mbps 1085.716
tx_rate_percent 5.641
throughput_rx_percent 5.641
frame_loss_percent 0.000
min_latency_ns 12140.000
max_latency_ns 523320.000
avg_latency_ns 57114.500
type rfc2544_throughput
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:28:49
start_time 2018-09-22 14:10:53
stop_time 2018-09-22 14:39:42
guest_loopback_app testpmd
4.2.8.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.19
%system 1.20
%guest 0.00
%CPU 1.39
CPU
minflt/s 0.23
majflt/s 0.00
VSZ 3364104
RSS 7767
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5900
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 1.04
%guest 198.99
%CPU 200.07
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2868554
RSS 66638
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 12.57
kB_ccwr/s 0.00
4.2.8.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.8.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 320283.062
tx_rate_mbps Unknown
throughput_rx_mbps 2623.759
tx_rate_percent 13.375
throughput_rx_percent 13.375
frame_loss_percent 0.000
min_latency_ns 17940.000
max_latency_ns 477800.000
avg_latency_ns 66919.500
type rfc2544_throughput
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:28:49
start_time 2018-09-22 14:10:53
stop_time 2018-09-22 14:39:42
guest_loopback_app testpmd
4.2.8.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.19
%system 1.20
%guest 0.00
%CPU 1.39
CPU
minflt/s 0.23
majflt/s 0.00
VSZ 3364104
RSS 7767
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5900
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 1.04
%guest 198.99
%CPU 200.07
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2868554
RSS 66638
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 12.57
kB_ccwr/s 0.00
4.2.8.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.8.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 280270.138
tx_rate_mbps Unknown
throughput_rx_mbps 3403.601
tx_rate_percent 17.242
throughput_rx_percent 17.242
frame_loss_percent 0.000
min_latency_ns 13140.000
max_latency_ns 646820.000
avg_latency_ns 64948.500
type rfc2544_throughput
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:28:49
start_time 2018-09-22 14:10:53
stop_time 2018-09-22 14:39:42
guest_loopback_app testpmd
4.2.8.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.19
%system 1.20
%guest 0.00
%CPU 1.39
CPU
minflt/s 0.23
majflt/s 0.00
VSZ 3364104
RSS 7767
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44320
RSS 5900
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 1.04
%guest 198.99
%CPU 200.07
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2868554
RSS 66638
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 12.57
kB_ccwr/s 0.00
4.2.8.7. Anomalies

No anomalies were detected during the course of this test.

4.2.8.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.3. Rationale for decisions

The tests conducted do not have pass/fail/conditional-pass criteria. The test is simply conducted and the results are reported.

4.4. Conclusions and recommendations

The test results are stable. The vsperf CI jobs that were used to obtain the results can be found at https://artifacts.opnfv.org/logs/vswitchperf/intel-pod12/2018-09-22_16-06-51/vswitchperf_logs_2018-09-22_16-06-51.tar.gz.

4.5. General

4.5.1. Glossary

  • NFV - Network Function Virtualization
  • Mbps - 1,000,000bps

4.5.2. Document change procedures and history

Document ID Author Date Modified
OPNFV_vswitchperf_LTR_ver_1.0_Jan_15_CN_DRAFT Christopher Nolan 23/01/2015
OPNFV_vswitchperf_LTR_ver_1.1_Jan_15_CN_DRAFT Christopher Nolan 28/01/2015

5. Performance report for VppDpdkVhost

5.1. Introduction

The objective of the OPNFV project titled “Characterise vSwitch Performance for Telco NFV Use Cases”, is to evaluate a virtual switch to identify its suitability for a Telco Network Function Virtualization (NFV) environment. As well as this, the project aims to identify any gaps or bottlenecks in order to drive architectural changes to improve virtual switch performance and determinism. The purpose of this document is to summarize the results of the tests carried out on the virtual switch in the Network Function Virtualization Infrastructure (NFVI) and, from these results, provide evaluations and recommendations for the virtual switch. Test results will be outlined in details-of-LTR, preceded by the document-identifier and the scope and references).

This document is currently in draft form.

5.1.1. Document identifier

The document id will be used to uniquely identify versions of the LTR. The format for the document id will be: OPNFV_vswitchperf_LTR_rel_STATUS, the status is one of: DRAFT, REVIEWED, CORRECTED or FINAL. The document id for this version of the LTR is: OPNFV_vswitchperf_LTR_Brahmaputra_DRAFT.

5.1.2. Scope

The scope of this report is to detail the results of the tests that have been performed on the virtual switch. This report will also evaluate the results of these tests and, based on these evaluations, provide recommendations on the suitability of the virtual switch for use in a Telco NFV environment.

5.2. Details of the Level Test Report

This section provides a test-results-overview. Also included are the rationale and the conclusions.

5.2.1. Test ID: PHY2PHY_TPUT_VPP

5.2.1.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 665dcb2b00632384762f2cf9754b2e2cdce6a69d

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: VppDpdkVhost, Version: v17.07-release, GIT tag: NA

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: phy2phy_tput_vpp
  • Description: VPP: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: p2p
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
5.2.1.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

5.2.1.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 20784604.290
tx_rate_mbps Unknown
throughput_rx_mbps 10641.717
tx_rate_percent 69.836
throughput_rx_percent 69.836
frame_loss_percent 0.000
min_latency_ns 5600.000
max_latency_ns 70480.000
avg_latency_ns 31453.000
type rfc2544_throughput
packet_size 64
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:22:06
start_time 2018-09-22 16:37:24
stop_time 2018-09-22 16:59:30
5.2.1.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.15
%system 0.29
%guest 0.00
%CPU 600.44
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 3937580
RSS 50500
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.1.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

5.2.1.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 16630607.391
tx_rate_mbps Unknown
throughput_rx_mbps 17029.742
tx_rate_percent 98.453
throughput_rx_percent 98.453
frame_loss_percent 0.000
min_latency_ns 5540.000
max_latency_ns 34920.000
avg_latency_ns 14277.500
type rfc2544_throughput
packet_size 128
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:22:06
start_time 2018-09-22 16:37:24
stop_time 2018-09-22 16:59:30
5.2.1.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.15
%system 0.29
%guest 0.00
%CPU 600.44
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 3937580
RSS 50500
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.1.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

5.2.1.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699185.367
tx_rate_mbps Unknown
throughput_rx_mbps 19247.863
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 5560.000
max_latency_ns 50380.000
avg_latency_ns 35255.500
type rfc2544_throughput
packet_size 512
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:22:06
start_time 2018-09-22 16:37:24
stop_time 2018-09-22 16:59:30
5.2.1.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.15
%system 0.29
%guest 0.00
%CPU 600.44
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 3937580
RSS 50500
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.1.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

5.2.1.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394614.588
tx_rate_mbps Unknown
throughput_rx_mbps 19616.683
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 6480.000
max_latency_ns 34860.000
avg_latency_ns 22515.500
type rfc2544_throughput
packet_size 1024
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:22:06
start_time 2018-09-22 16:37:24
stop_time 2018-09-22 16:59:30
5.2.1.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.15
%system 0.29
%guest 0.00
%CPU 600.44
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 3937580
RSS 50500
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.1.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

5.2.1.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625471.945
tx_rate_mbps Unknown
throughput_rx_mbps 19739.731
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 6700.000
max_latency_ns 37400.000
avg_latency_ns 25462.000
type rfc2544_throughput
packet_size 1518
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:22:06
start_time 2018-09-22 16:37:24
stop_time 2018-09-22 16:59:30
5.2.1.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.15
%system 0.29
%guest 0.00
%CPU 600.44
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 3937580
RSS 50500
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.1.7. Anomalies

No anomalies were detected during the course of this test.

5.2.1.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

5.2.2. Test ID: PVP_TPUT_VPP

5.2.2.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: 665dcb2b00632384762f2cf9754b2e2cdce6a69d

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: VppDpdkVhost, Version: v17.07-release, GIT tag: NA

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

  • VNF: QemuDpdkVhostUser, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-16.04_trex_20180209.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvp_tput_vpp
  • Description: VPP: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvp
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
5.2.2.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

5.2.2.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 527802.857
tx_rate_mbps Unknown
throughput_rx_mbps 270.235
tx_rate_percent 1.773
throughput_rx_percent 1.773
frame_loss_percent 0.001
min_latency_ns 5080.000
max_latency_ns 1179160.000
avg_latency_ns 5392.500
type rfc2544_throughput
packet_size 64
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:29:09
start_time 2018-09-22 17:32:54
stop_time 2018-09-22 18:02:03
guest_loopback_app testpmd
5.2.2.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.05
%system 0.04
%guest 200.00
%CPU 200.09
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2764245
RSS 66920
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.02
kB_ccwr/s 0.00
Process: vpp_main
Statistic Value
%usr 600.16
%system 0.29
%guest 0.00
%CPU 600.45
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 10229036
RSS 50936
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.2.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

5.2.2.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 6440042.070
tx_rate_mbps Unknown
throughput_rx_mbps 6594.603
tx_rate_percent 38.125
throughput_rx_percent 38.125
frame_loss_percent 0.000
min_latency_ns 6840.000
max_latency_ns 53160.000
avg_latency_ns 9453.500
type rfc2544_throughput
packet_size 128
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:29:09
start_time 2018-09-22 17:32:54
stop_time 2018-09-22 18:02:03
guest_loopback_app testpmd
5.2.2.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.05
%system 0.04
%guest 200.00
%CPU 200.09
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2764245
RSS 66920
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.02
kB_ccwr/s 0.00
Process: vpp_main
Statistic Value
%usr 600.16
%system 0.29
%guest 0.00
%CPU 600.45
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 10229036
RSS 50936
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.2.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

5.2.2.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699154.102
tx_rate_mbps Unknown
throughput_rx_mbps 19247.735
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 8800.000
max_latency_ns 71700.000
avg_latency_ns 56792.000
type rfc2544_throughput
packet_size 512
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:29:09
start_time 2018-09-22 17:32:54
stop_time 2018-09-22 18:02:03
guest_loopback_app testpmd
5.2.2.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.05
%system 0.04
%guest 200.00
%CPU 200.09
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2764245
RSS 66920
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.02
kB_ccwr/s 0.00
Process: vpp_main
Statistic Value
%usr 600.16
%system 0.29
%guest 0.00
%CPU 600.45
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 10229036
RSS 50936
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.2.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

5.2.2.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394571.869
tx_rate_mbps Unknown
throughput_rx_mbps 19616.333
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 9780.000
max_latency_ns 108880.000
avg_latency_ns 73766.500
type rfc2544_throughput
packet_size 1024
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:29:09
start_time 2018-09-22 17:32:54
stop_time 2018-09-22 18:02:03
guest_loopback_app testpmd
5.2.2.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.05
%system 0.04
%guest 200.00
%CPU 200.09
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2764245
RSS 66920
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.02
kB_ccwr/s 0.00
Process: vpp_main
Statistic Value
%usr 600.16
%system 0.29
%guest 0.00
%CPU 600.45
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 10229036
RSS 50936
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.2.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

5.2.2.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1424334.775
tx_rate_mbps Unknown
throughput_rx_mbps 17297.122
tx_rate_percent 87.625
throughput_rx_percent 87.625
frame_loss_percent 0.000
min_latency_ns 8120.000
max_latency_ns 42680.000
avg_latency_ns 10311.000
type rfc2544_throughput
packet_size 1518
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:29:09
start_time 2018-09-22 17:32:54
stop_time 2018-09-22 18:02:03
guest_loopback_app testpmd
5.2.2.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.05
%system 0.04
%guest 200.00
%CPU 200.09
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2764245
RSS 66920
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.02
kB_ccwr/s 0.00
Process: vpp_main
Statistic Value
%usr 600.16
%system 0.29
%guest 0.00
%CPU 600.45
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 10229036
RSS 50936
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.2.7. Anomalies

No anomalies were detected during the course of this test.

5.2.2.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

5.2.3. Test ID: PVVP_TPUT_VPP

5.2.3.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvvp

  • vswitchperf: GIT tag: 665dcb2b00632384762f2cf9754b2e2cdce6a69d

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: VppDpdkVhost, Version: v17.07-release, GIT tag: NA

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

  • VNF: QemuDpdkVhostUser, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-16.04_trex_20180209.qcow2
    • /home/jenkins/vloop-vnf-ubuntu-16.04_trex_20180209.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvvp_tput_vpp
  • Description: VPP: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvvp
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
5.2.3.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

5.2.3.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 527790.747
tx_rate_mbps Unknown
throughput_rx_mbps 270.229
tx_rate_percent 1.773
throughput_rx_percent 1.773
frame_loss_percent 0.003
min_latency_ns 10540.000
max_latency_ns 2594220.000
avg_latency_ns 20877.500
type rfc2544_throughput
packet_size 64
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:30:18
start_time 2018-09-22 18:06:32
stop_time 2018-09-22 18:36:50
guest_loopback_app testpmd testpmd
5.2.3.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.20
%system 0.39
%guest 200.25
%CPU 200.85
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2848151
RSS 68943
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 12.35
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.09
%system 0.19
%guest 200.12
%CPU 200.40
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2820412
RSS 60316
%MEM 0.09
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: vpp_main
Statistic Value
%usr 600.43
%system 0.61
%guest 0.00
%CPU 601.04
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 16520492
RSS 51364
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.3.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

5.2.3.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 299545.283
tx_rate_mbps Unknown
throughput_rx_mbps 306.734
tx_rate_percent 1.773
throughput_rx_percent 1.773
frame_loss_percent 0.007
min_latency_ns 10500.000
max_latency_ns 3423180.000
avg_latency_ns 19746.000
type rfc2544_throughput
packet_size 128
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:30:18
start_time 2018-09-22 18:06:32
stop_time 2018-09-22 18:36:50
guest_loopback_app testpmd testpmd
5.2.3.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.20
%system 0.39
%guest 200.25
%CPU 200.85
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2848151
RSS 68943
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 12.35
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.09
%system 0.19
%guest 200.12
%CPU 200.40
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2820412
RSS 60316
%MEM 0.09
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: vpp_main
Statistic Value
%usr 600.43
%system 0.61
%guest 0.00
%CPU 601.04
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 16520492
RSS 51364
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.3.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

5.2.3.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1791581.938
tx_rate_mbps Unknown
throughput_rx_mbps 7338.320
tx_rate_percent 38.125
throughput_rx_percent 38.125
frame_loss_percent 0.000
min_latency_ns 20460.000
max_latency_ns 119880.000
avg_latency_ns 38354.500
type rfc2544_throughput
packet_size 512
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:30:18
start_time 2018-09-22 18:06:32
stop_time 2018-09-22 18:36:50
guest_loopback_app testpmd testpmd
5.2.3.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.20
%system 0.39
%guest 200.25
%CPU 200.85
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2848151
RSS 68943
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 12.35
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.09
%system 0.19
%guest 200.12
%CPU 200.40
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2820412
RSS 60316
%MEM 0.09
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: vpp_main
Statistic Value
%usr 600.43
%system 0.61
%guest 0.00
%CPU 601.04
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 16520492
RSS 51364
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.3.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

5.2.3.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1801923.997
tx_rate_mbps Unknown
throughput_rx_mbps 14761.361
tx_rate_percent 75.250
throughput_rx_percent 75.250
frame_loss_percent 0.000
min_latency_ns 36660.000
max_latency_ns 161520.000
avg_latency_ns 57410.500
type rfc2544_throughput
packet_size 1024
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:30:18
start_time 2018-09-22 18:06:32
stop_time 2018-09-22 18:36:50
guest_loopback_app testpmd testpmd
5.2.3.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.20
%system 0.39
%guest 200.25
%CPU 200.85
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2848151
RSS 68943
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 12.35
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.09
%system 0.19
%guest 200.12
%CPU 200.40
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2820412
RSS 60316
%MEM 0.09
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: vpp_main
Statistic Value
%usr 600.43
%system 0.61
%guest 0.00
%CPU 601.04
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 16520492
RSS 51364
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.3.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

5.2.3.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 619716.871
tx_rate_mbps Unknown
throughput_rx_mbps 7525.842
tx_rate_percent 38.125
throughput_rx_percent 38.125
frame_loss_percent 0.000
min_latency_ns 18440.000
max_latency_ns 121620.000
avg_latency_ns 30607.000
type rfc2544_throughput
packet_size 1518
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:30:18
start_time 2018-09-22 18:06:32
stop_time 2018-09-22 18:36:50
guest_loopback_app testpmd testpmd
5.2.3.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.20
%system 0.39
%guest 200.25
%CPU 200.85
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2848151
RSS 68943
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 12.35
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.09
%system 0.19
%guest 200.12
%CPU 200.40
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2820412
RSS 60316
%MEM 0.09
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: vpp_main
Statistic Value
%usr 600.43
%system 0.61
%guest 0.00
%CPU 601.04
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 16520492
RSS 51364
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.3.7. Anomalies

No anomalies were detected during the course of this test.

5.2.3.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

5.2.4. Test ID: PVVP_BACK2BACK_VPP

5.2.4.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvvp

  • vswitchperf: GIT tag: 665dcb2b00632384762f2cf9754b2e2cdce6a69d

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: VppDpdkVhost, Version: v17.07-release, GIT tag: NA

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

  • VNF: QemuDpdkVhostUser, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-16.04_trex_20180209.qcow2
    • /home/jenkins/vloop-vnf-ubuntu-16.04_trex_20180209.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvvp_back2back_vpp
  • Description: VPP: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: pvvp
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
5.2.4.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

5.2.4.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544_back2back
packet_size 64
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:47:30
start_time 2018-09-22 18:36:50
stop_time 2018-09-22 19:24:21
guest_loopback_app testpmd testpmd
5.2.4.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.09
%system 0.09
%guest 200.04
%CPU 200.22
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2800767
RSS 67059
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 7.90
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.08
%system 0.07
%guest 200.03
%CPU 200.19
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2811160
RSS 67300
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.02
kB_ccwr/s 0.00
Process: vpp_main
Statistic Value
%usr 600.17
%system 0.29
%guest 0.00
%CPU 600.46
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 16520492
RSS 51112
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.4.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

5.2.4.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544_back2back
packet_size 128
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:47:30
start_time 2018-09-22 18:36:50
stop_time 2018-09-22 19:24:21
guest_loopback_app testpmd testpmd
5.2.4.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.09
%system 0.09
%guest 200.04
%CPU 200.22
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2800767
RSS 67059
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 7.90
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.08
%system 0.07
%guest 200.03
%CPU 200.19
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2811160
RSS 67300
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.02
kB_ccwr/s 0.00
Process: vpp_main
Statistic Value
%usr 600.17
%system 0.29
%guest 0.00
%CPU 600.46
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 16520492
RSS 51112
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.4.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

5.2.4.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 537
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:47:30
start_time 2018-09-22 18:36:50
stop_time 2018-09-22 19:24:21
guest_loopback_app testpmd testpmd
5.2.4.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.09
%system 0.09
%guest 200.04
%CPU 200.22
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2800767
RSS 67059
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 7.90
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.08
%system 0.07
%guest 200.03
%CPU 200.19
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2811160
RSS 67300
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.02
kB_ccwr/s 0.00
Process: vpp_main
Statistic Value
%usr 600.17
%system 0.29
%guest 0.00
%CPU 600.46
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 16520492
RSS 51112
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.4.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

5.2.4.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 8979885
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:47:30
start_time 2018-09-22 18:36:50
stop_time 2018-09-22 19:24:21
guest_loopback_app testpmd testpmd
5.2.4.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.09
%system 0.09
%guest 200.04
%CPU 200.22
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2800767
RSS 67059
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 7.90
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.08
%system 0.07
%guest 200.03
%CPU 200.19
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2811160
RSS 67300
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.02
kB_ccwr/s 0.00
Process: vpp_main
Statistic Value
%usr 600.17
%system 0.29
%guest 0.00
%CPU 600.46
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 16520492
RSS 51112
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.4.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

5.2.4.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 12191157
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:47:30
start_time 2018-09-22 18:36:50
stop_time 2018-09-22 19:24:21
guest_loopback_app testpmd testpmd
5.2.4.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.09
%system 0.09
%guest 200.04
%CPU 200.22
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2800767
RSS 67059
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 7.90
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.08
%system 0.07
%guest 200.03
%CPU 200.19
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2811160
RSS 67300
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.02
kB_ccwr/s 0.00
Process: vpp_main
Statistic Value
%usr 600.17
%system 0.29
%guest 0.00
%CPU 600.46
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 16520492
RSS 51112
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.4.7. Anomalies

No anomalies were detected during the course of this test.

5.2.4.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

5.2.5. Test ID: PHY2PHY_BACK2BACK_VPP

5.2.5.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 665dcb2b00632384762f2cf9754b2e2cdce6a69d

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: VppDpdkVhost, Version: v17.07-release, GIT tag: NA

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: phy2phy_back2back_vpp
  • Description: VPP: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: p2p
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
5.2.5.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

5.2.5.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 23841
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 64
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:33:23
start_time 2018-09-22 16:59:31
stop_time 2018-09-22 17:32:54
5.2.5.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.15
%system 0.28
%guest 0.00
%CPU 600.43
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 3937580
RSS 50684
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.5.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

5.2.5.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 247439
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 128
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:33:23
start_time 2018-09-22 16:59:31
stop_time 2018-09-22 17:32:54
5.2.5.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.15
%system 0.28
%guest 0.00
%CPU 600.43
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 3937580
RSS 50684
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.5.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

5.2.5.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 70488721
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:33:23
start_time 2018-09-22 16:59:31
stop_time 2018-09-22 17:32:54
5.2.5.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.15
%system 0.28
%guest 0.00
%CPU 600.43
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 3937580
RSS 50684
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.5.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

5.2.5.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 35919540
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:33:23
start_time 2018-09-22 16:59:31
stop_time 2018-09-22 17:32:54
5.2.5.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.15
%system 0.28
%guest 0.00
%CPU 600.43
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 3937580
RSS 50684
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.5.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

5.2.5.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 24382314
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:33:23
start_time 2018-09-22 16:59:31
stop_time 2018-09-22 17:32:54
5.2.5.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.15
%system 0.28
%guest 0.00
%CPU 600.43
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 3937580
RSS 50684
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.5.7. Anomalies

No anomalies were detected during the course of this test.

5.2.5.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

5.3. Rationale for decisions

The tests conducted do not have pass/fail/conditional-pass criteria. The test is simply conducted and the results are reported.

5.4. Conclusions and recommendations

The test results are stable. The vsperf CI jobs that were used to obtain the results can be found at https://artifacts.opnfv.org/logs/vswitchperf/intel-pod12/2018-09-22_16-06-51/vswitchperf_logs_2018-09-22_16-06-51.tar.gz.

5.5. General

5.5.1. Glossary

  • NFV - Network Function Virtualization
  • Mbps - 1,000,000bps

5.5.2. Document change procedures and history

Document ID Author Date Modified
OPNFV_vswitchperf_LTR_ver_1.0_Jan_15_CN_DRAFT Christopher Nolan 23/01/2015
OPNFV_vswitchperf_LTR_ver_1.1_Jan_15_CN_DRAFT Christopher Nolan 28/01/2015

6. Performance report for none

6.1. Introduction

The objective of the OPNFV project titled “Characterise vSwitch Performance for Telco NFV Use Cases”, is to evaluate a virtual switch to identify its suitability for a Telco Network Function Virtualization (NFV) environment. As well as this, the project aims to identify any gaps or bottlenecks in order to drive architectural changes to improve virtual switch performance and determinism. The purpose of this document is to summarize the results of the tests carried out on the virtual switch in the Network Function Virtualization Infrastructure (NFVI) and, from these results, provide evaluations and recommendations for the virtual switch. Test results will be outlined in details-of-LTR, preceded by the document-identifier and the scope and references).

This document is currently in draft form.

6.1.1. Document identifier

The document id will be used to uniquely identify versions of the LTR. The format for the document id will be: OPNFV_vswitchperf_LTR_rel_STATUS, the status is one of: DRAFT, REVIEWED, CORRECTED or FINAL. The document id for this version of the LTR is: OPNFV_vswitchperf_LTR_Brahmaputra_DRAFT.

6.1.2. Scope

The scope of this report is to detail the results of the tests that have been performed on the virtual switch. This report will also evaluate the results of these tests and, based on these evaluations, provide recommendations on the suitability of the virtual switch for use in a Telco NFV environment.

6.2. Details of the Level Test Report

This section provides a test-results-overview. Also included are the rationale and the conclusions.

6.2.1. Test ID: PVP_TPUT

6.2.1.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.116-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
    • Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: 665dcb2b00632384762f2cf9754b2e2cdce6a69d

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: none, Version: None, GIT tag: None

  • VNF: QemuPciPassthrough, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvp_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvp
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
6.2.1.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

6.2.1.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 7663707.805
tx_rate_mbps Unknown
throughput_rx_mbps 3923.818
tx_rate_percent 25.750
throughput_rx_percent 25.750
frame_loss_percent 0.000
min_latency_ns 3600.000
max_latency_ns 27180.000
avg_latency_ns 3958.500
type rfc2544_throughput
packet_size 64
vswitch none
traffic_type udp
test_execution_time 00:23:53
start_time 2018-09-22 19:24:32
stop_time 2018-09-22 19:48:25
guest_loopback_app testpmd
6.2.1.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.01
%system 0.02
%guest 200.00
%CPU 200.03
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2847196
RSS 78291
%MEM 0.12
kB_rd/s 0.00
kB_wr/s 15.98
kB_ccwr/s 0.00
6.2.1.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

6.2.1.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 8530433.812
tx_rate_mbps Unknown
throughput_rx_mbps 8735.164
tx_rate_percent 50.500
throughput_rx_percent 50.500
frame_loss_percent 0.000
min_latency_ns 3820.000
max_latency_ns 27680.000
avg_latency_ns 4285.000
type rfc2544_throughput
packet_size 128
vswitch none
traffic_type udp
test_execution_time 00:23:53
start_time 2018-09-22 19:24:32
stop_time 2018-09-22 19:48:25
guest_loopback_app testpmd
6.2.1.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.01
%system 0.02
%guest 200.00
%CPU 200.03
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2847196
RSS 78291
%MEM 0.12
kB_rd/s 0.00
kB_wr/s 15.98
kB_ccwr/s 0.00
6.2.1.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

6.2.1.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699193.230
tx_rate_mbps Unknown
throughput_rx_mbps 19247.895
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 5060.000
max_latency_ns 44540.000
avg_latency_ns 27429.000
type rfc2544_throughput
packet_size 512
vswitch none
traffic_type udp
test_execution_time 00:23:53
start_time 2018-09-22 19:24:32
stop_time 2018-09-22 19:48:25
guest_loopback_app testpmd
6.2.1.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.01
%system 0.02
%guest 200.00
%CPU 200.03
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2847196
RSS 78291
%MEM 0.12
kB_rd/s 0.00
kB_wr/s 15.98
kB_ccwr/s 0.00
6.2.1.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

6.2.1.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394608.841
tx_rate_mbps Unknown
throughput_rx_mbps 19616.636
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 5400.000
max_latency_ns 42000.000
avg_latency_ns 27814.500
type rfc2544_throughput
packet_size 1024
vswitch none
traffic_type udp
test_execution_time 00:23:53
start_time 2018-09-22 19:24:32
stop_time 2018-09-22 19:48:25
guest_loopback_app testpmd
6.2.1.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.01
%system 0.02
%guest 200.00
%CPU 200.03
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2847196
RSS 78291
%MEM 0.12
kB_rd/s 0.00
kB_wr/s 15.98
kB_ccwr/s 0.00
6.2.1.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

6.2.1.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625470.087
tx_rate_mbps Unknown
throughput_rx_mbps 19739.709
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 5940.000
max_latency_ns 39580.000
avg_latency_ns 26692.000
type rfc2544_throughput
packet_size 1518
vswitch none
traffic_type udp
test_execution_time 00:23:53
start_time 2018-09-22 19:24:32
stop_time 2018-09-22 19:48:25
guest_loopback_app testpmd
6.2.1.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.01
%system 0.02
%guest 200.00
%CPU 200.03
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2847196
RSS 78291
%MEM 0.12
kB_rd/s 0.00
kB_wr/s 15.98
kB_ccwr/s 0.00
6.2.1.7. Anomalies

No anomalies were detected during the course of this test.

6.2.1.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

6.3. Rationale for decisions

The tests conducted do not have pass/fail/conditional-pass criteria. The test is simply conducted and the results are reported.

6.4. Conclusions and recommendations

The test results are stable. The vsperf CI jobs that were used to obtain the results can be found at https://artifacts.opnfv.org/logs/vswitchperf/intel-pod12/2018-09-22_16-06-51/vswitchperf_logs_2018-09-22_16-06-51.tar.gz.

6.5. General

6.5.1. Glossary

  • NFV - Network Function Virtualization
  • Mbps - 1,000,000bps

6.5.2. Document change procedures and history

Document ID Author Date Modified
OPNFV_vswitchperf_LTR_ver_1.0_Jan_15_CN_DRAFT Christopher Nolan 23/01/2015
OPNFV_vswitchperf_LTR_ver_1.1_Jan_15_CN_DRAFT Christopher Nolan 28/01/2015