VSPERF Results

1. VSPERF Test Scenarios

Predefined Tests suitable for automated execution with CI:

Test Definition
phy2phy_tput PacketLossRatio for Phy2Phy
back2back BackToBackFrames for Phy2Phy
phy2phy_tput_mod_vlan PacketLossRatioFrameModification for Phy2Phy
phy2phy_cont Phy2Phy blast vswitch at x% TX rate and measure throughput
pvp_cont PVP blast vswitch at x% TX rate and measure throughput
pvvp_cont PVVP blast vswitch at x% TX rate and measure throughput
phy2phy_scalability Scalability0PacketLoss for Phy2Phy
pvp_tput PacketLossRatio for PVP
pvp_back2back BackToBackFrames for PVP
pvvp_tput PacketLossRatio for PVVP
pvvp_back2back BackToBackFrames for PVVP
phy2phy_cpu_load CPU0PacketLoss for Phy2Phy
phy2phy_mem_load Same as CPU0PacketLoss but using a memory intensive app

Deployment topologies:

  • Phy2Phy: Physical port -> vSwitch -> Physical port.
  • PVP: Physical port -> vSwitch -> VNF -> vSwitch -> Physical port.
  • PVVP: Physical port -> vSwitch -> VNF -> vSwitch -> VNF -> vSwitch -> Physical port.

Loopback applications in the Guest:

Supported traffic generators:

  • Spirent Testcenter
  • Ixia: IxOS and IxNet.
  • Xena
  • MoonGen
  • Dummy
  • T-Rex

2. OPNFV Test Results

VSPERF CI jobs are run daily and sample results can be found at https://wiki.opnfv.org/display/vsperf/Vsperf+Results

Testcase names shown in the dashboard are combination of orignal testcase name from VSPERF framework and indication of used vswitch.

Example:

Testcase phy2phy_tput is executed for three vSwitch types: OvsDpdkVhost, OvsVanilla and VppDpdkVhost. In this case, following testcase names will be used in the dashboard: phy2phy_tput_ovsdpdkvhost, phy2phy_tput_ovsvanilla and phy2phy_tput_vppdpdkvhost.

In case of RFC2544 Throughput test, the recorded metric is FPS (frames per second) without packet loss. For RFC2544 Back2Back test, the recorded metric is back-to-back value (number of frames) without packet loss.

The loopback application in the VNF used for PVP and PVVP scenarios was DPDK testpmd.

Guest interface types are vhost-user for OvsDpdkVhost and VppDpdkVhost and virtio-net for OvsVanilla.

3. Performance report for OvsDpdkVhost

3.1. Introduction

The objective of the OPNFV project titled “Characterise vSwitch Performance for Telco NFV Use Cases”, is to evaluate a virtual switch to identify its suitability for a Telco Network Function Virtualization (NFV) environment. As well as this, the project aims to identify any gaps or bottlenecks in order to drive architectural changes to improve virtual switch performance and determinism. The purpose of this document is to summarize the results of the tests carried out on the virtual switch in the Network Function Virtualization Infrastructure (NFVI) and, from these results, provide evaluations and recommendations for the virtual switch. Test results will be outlined in details-of-LTR, preceded by the document-identifier and the scope and references).

This document is currently in draft form.

3.1.1. Document identifier

The document id will be used to uniquely identify versions of the LTR. The format for the document id will be: OPNFV_vswitchperf_LTR_rel_STATUS, the status is one of: DRAFT, REVIEWED, CORRECTED or FINAL. The document id for this version of the LTR is: OPNFV_vswitchperf_LTR_Brahmaputra_DRAFT.

3.1.2. Scope

The scope of this report is to detail the results of the tests that have been performed on the virtual switch. This report will also evaluate the results of these tests and, based on these evaluations, provide recommendations on the suitability of the virtual switch for use in a Telco NFV environment.

3.2. Details of the Level Test Report

This section provides a test-results-overview. Also included are the rationale and the conclusions.

3.2.1. Test ID: BACK2BACK

3.2.1.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 7440717327dabe7ae37f99cb060bce43a7878096

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: p2p
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
3.2.1.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.1.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 25970
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 64
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:29:53
start_time 2018-04-23 14:36:07
stop_time 2018-04-23 15:06:01
3.2.1.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.04
%system 0.02
%guest 0.00
%CPU 200.16
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056776
RSS 43035
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6660
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.1.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 253378378
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 128
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:29:53
start_time 2018-04-23 14:36:07
stop_time 2018-04-23 15:06:01
3.2.1.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.04
%system 0.02
%guest 0.00
%CPU 200.16
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056776
RSS 43035
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6660
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.1.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 70488721
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:29:53
start_time 2018-04-23 14:36:07
stop_time 2018-04-23 15:06:01
3.2.1.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.04
%system 0.02
%guest 0.00
%CPU 200.16
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056776
RSS 43035
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6660
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.1.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 35919540
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:29:53
start_time 2018-04-23 14:36:07
stop_time 2018-04-23 15:06:01
3.2.1.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.04
%system 0.02
%guest 0.00
%CPU 200.16
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056776
RSS 43035
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6660
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.1.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 24382314
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:29:53
start_time 2018-04-23 14:36:07
stop_time 2018-04-23 15:06:01
3.2.1.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.04
%system 0.02
%guest 0.00
%CPU 200.16
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056776
RSS 43035
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6660
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.1.7. Anomalies

No anomalies were detected during the course of this test.

3.2.1.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.2. Test ID: PHY2PHY_SCALABILITY

3.2.2.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 7440717327dabe7ae37f99cb060bce43a7878096

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: phy2phy_scalability
  • Description: LTD.Scalability.Flows.RFC2544.0PacketLoss
  • Deployment: p2p
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
3.2.2.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.2.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 18252465.894
tx_rate_mbps Unknown
throughput_rx_mbps 9345.263
tx_rate_percent 61.328
throughput_rx_percent 61.328
frame_loss_percent 0.000
min_latency_ns 5020.000
max_latency_ns 40060.000
avg_latency_ns 11268.000
type rfc2544_throughput
packet_size 64
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:16:17
start_time 2018-04-23 15:33:38
stop_time 2018-04-23 15:49:55
stream_count 8000
match_type L4
pre-installed_flows No
3.2.2.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.04
%system 0.02
%guest 0.00
%CPU 200.08
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 5056776
RSS 43014
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6872
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.2.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.2.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 16761271.865
tx_rate_mbps Unknown
throughput_rx_mbps 17163.542
tx_rate_percent 99.227
throughput_rx_percent 99.227
frame_loss_percent 0.000
min_latency_ns 4940.000
max_latency_ns 40400.000
avg_latency_ns 12810.500
type rfc2544_throughput
packet_size 128
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:16:17
start_time 2018-04-23 15:33:38
stop_time 2018-04-23 15:49:55
stream_count 8000
match_type L4
pre-installed_flows No
3.2.2.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.04
%system 0.02
%guest 0.00
%CPU 200.08
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 5056776
RSS 43014
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6872
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.2.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.2.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699200.937
tx_rate_mbps Unknown
throughput_rx_mbps 19247.927
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 17820.000
max_latency_ns 50040.000
avg_latency_ns 38558.500
type rfc2544_throughput
packet_size 512
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:16:17
start_time 2018-04-23 15:33:38
stop_time 2018-04-23 15:49:55
stream_count 8000
match_type L4
pre-installed_flows No
3.2.2.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.04
%system 0.02
%guest 0.00
%CPU 200.08
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 5056776
RSS 43014
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6872
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.2.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.2.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394615.769
tx_rate_mbps Unknown
throughput_rx_mbps 19616.692
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 13580.000
max_latency_ns 40060.000
avg_latency_ns 29195.000
type rfc2544_throughput
packet_size 1024
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:16:17
start_time 2018-04-23 15:33:38
stop_time 2018-04-23 15:49:55
stream_count 8000
match_type L4
pre-installed_flows No
3.2.2.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.04
%system 0.02
%guest 0.00
%CPU 200.08
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 5056776
RSS 43014
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6872
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.2.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.2.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625474.779
tx_rate_mbps Unknown
throughput_rx_mbps 19739.766
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 15700.000
max_latency_ns 40260.000
avg_latency_ns 30068.500
type rfc2544_throughput
packet_size 1518
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:16:17
start_time 2018-04-23 15:33:38
stop_time 2018-04-23 15:49:55
stream_count 8000
match_type L4
pre-installed_flows No
3.2.2.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.04
%system 0.02
%guest 0.00
%CPU 200.08
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 5056776
RSS 43014
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6872
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.2.7. Anomalies

No anomalies were detected during the course of this test.

3.2.2.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.3. Test ID: PVVP_TPUT

3.2.3.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvvp

  • vswitchperf: GIT tag: 7440717327dabe7ae37f99cb060bce43a7878096

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

  • VNF: QemuDpdkVhostUser, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvvp_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvvp
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
3.2.3.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.3.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 5822177.907
tx_rate_mbps Unknown
throughput_rx_mbps 2980.955
tx_rate_percent 19.563
throughput_rx_percent 19.563
frame_loss_percent 0.000
min_latency_ns 7200.000
max_latency_ns 61240.000
avg_latency_ns 9573.000
type rfc2544_throughput
packet_size 64
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:18:05
start_time 2018-04-23 16:32:35
stop_time 2018-04-23 16:50:40
guest_loopback_app testpmd testpmd
3.2.3.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 600.04
%system 0.02
%guest 0.00
%CPU 600.17
CPU
minflt/s 0.07
majflt/s 0.00
VSZ 17713420
RSS 43080
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6484
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 0.02
%guest 200.02
%CPU 200.06
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2815252
RSS 67031
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 21.84
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 0.03
%guest 200.02
%CPU 200.07
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2852260
RSS 67972
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.05
kB_ccwr/s 0.00
3.2.3.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.3.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 6570689.956
tx_rate_mbps Unknown
throughput_rx_mbps 6728.387
tx_rate_percent 38.898
throughput_rx_percent 38.898
frame_loss_percent 0.000
min_latency_ns 9500.000
max_latency_ns 63780.000
avg_latency_ns 12717.000
type rfc2544_throughput
packet_size 128
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:18:05
start_time 2018-04-23 16:32:35
stop_time 2018-04-23 16:50:40
guest_loopback_app testpmd testpmd
3.2.3.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 600.04
%system 0.02
%guest 0.00
%CPU 600.17
CPU
minflt/s 0.07
majflt/s 0.00
VSZ 17713420
RSS 43080
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6484
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 0.02
%guest 200.02
%CPU 200.06
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2815252
RSS 67031
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 21.84
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 0.03
%guest 200.02
%CPU 200.07
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2852260
RSS 67972
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.05
kB_ccwr/s 0.00
3.2.3.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.3.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699155.950
tx_rate_mbps Unknown
throughput_rx_mbps 19247.743
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 33760.000
max_latency_ns 97600.000
avg_latency_ns 85810.500
type rfc2544_throughput
packet_size 512
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:18:05
start_time 2018-04-23 16:32:35
stop_time 2018-04-23 16:50:40
guest_loopback_app testpmd testpmd
3.2.3.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 600.04
%system 0.02
%guest 0.00
%CPU 600.17
CPU
minflt/s 0.07
majflt/s 0.00
VSZ 17713420
RSS 43080
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6484
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 0.02
%guest 200.02
%CPU 200.06
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2815252
RSS 67031
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 21.84
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 0.03
%guest 200.02
%CPU 200.07
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2852260
RSS 67972
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.05
kB_ccwr/s 0.00
3.2.3.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.3.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394590.036
tx_rate_mbps Unknown
throughput_rx_mbps 19616.482
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 32060.000
max_latency_ns 94800.000
avg_latency_ns 79210.000
type rfc2544_throughput
packet_size 1024
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:18:05
start_time 2018-04-23 16:32:35
stop_time 2018-04-23 16:50:40
guest_loopback_app testpmd testpmd
3.2.3.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 600.04
%system 0.02
%guest 0.00
%CPU 600.17
CPU
minflt/s 0.07
majflt/s 0.00
VSZ 17713420
RSS 43080
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6484
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 0.02
%guest 200.02
%CPU 200.06
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2815252
RSS 67031
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 21.84
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 0.03
%guest 200.02
%CPU 200.07
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2852260
RSS 67972
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.05
kB_ccwr/s 0.00
3.2.3.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.3.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625464.734
tx_rate_mbps Unknown
throughput_rx_mbps 19739.644
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 33060.000
max_latency_ns 83920.000
avg_latency_ns 65819.500
type rfc2544_throughput
packet_size 1518
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:18:05
start_time 2018-04-23 16:32:35
stop_time 2018-04-23 16:50:40
guest_loopback_app testpmd testpmd
3.2.3.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 600.04
%system 0.02
%guest 0.00
%CPU 600.17
CPU
minflt/s 0.07
majflt/s 0.00
VSZ 17713420
RSS 43080
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6484
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 0.02
%guest 200.02
%CPU 200.06
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2815252
RSS 67031
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 21.84
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 0.03
%guest 200.02
%CPU 200.07
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2852260
RSS 67972
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.05
kB_ccwr/s 0.00
3.2.3.7. Anomalies

No anomalies were detected during the course of this test.

3.2.3.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.4. Test ID: PHY2PHY_TPUT_MOD_VLAN

3.2.4.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 7440717327dabe7ae37f99cb060bce43a7878096

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: phy2phy_tput_mod_vlan
  • Description: LTD.Throughput.RFC2544.PacketLossRatioFrameModification
  • Deployment: p2p
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
3.2.4.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.4.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 16410990.160
tx_rate_mbps Unknown
throughput_rx_mbps 8927.579
tx_rate_percent 55.141
throughput_rx_percent 57.767
frame_loss_percent 0.000
min_latency_ns 4840.000
max_latency_ns 38460.000
avg_latency_ns 11189.000
type rfc2544_throughput
packet_size 64
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:27:36
start_time 2018-04-23 15:06:01
stop_time 2018-04-23 15:33:38
3.2.4.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.04
%system 0.02
%guest 0.00
%CPU 200.08
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056776
RSS 43027
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6624
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.4.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.4.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 16369327.372
tx_rate_mbps Unknown
throughput_rx_mbps 17286.010
tx_rate_percent 96.906
throughput_rx_percent 99.526
frame_loss_percent 0.000
min_latency_ns 5260.000
max_latency_ns 40780.000
avg_latency_ns 15901.000
type rfc2544_throughput
packet_size 128
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:27:36
start_time 2018-04-23 15:06:01
stop_time 2018-04-23 15:33:38
3.2.4.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.04
%system 0.02
%guest 0.00
%CPU 200.08
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056776
RSS 43027
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6624
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.4.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.4.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4662904.595
tx_rate_mbps Unknown
throughput_rx_mbps 19248.470
tx_rate_percent 99.227
throughput_rx_percent 99.973
frame_loss_percent 0.000
min_latency_ns 5520.000
max_latency_ns 30380.000
avg_latency_ns 7014.000
type rfc2544_throughput
packet_size 512
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:27:36
start_time 2018-04-23 15:06:01
stop_time 2018-04-23 15:33:38
3.2.4.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.04
%system 0.02
%guest 0.00
%CPU 200.08
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056776
RSS 43027
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6624
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.4.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.4.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2376116.982
tx_rate_mbps Unknown
throughput_rx_mbps 19541.186
tx_rate_percent 99.227
throughput_rx_percent 99.607
frame_loss_percent 0.000
min_latency_ns 6060.000
max_latency_ns 20360.000
avg_latency_ns 7440.500
type rfc2544_throughput
packet_size 1024
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:27:36
start_time 2018-04-23 15:06:01
stop_time 2018-04-23 15:33:38
3.2.4.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.04
%system 0.02
%guest 0.00
%CPU 200.08
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056776
RSS 43027
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6624
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.4.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.4.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1612916.353
tx_rate_mbps Unknown
throughput_rx_mbps 19638.870
tx_rate_percent 99.227
throughput_rx_percent 99.485
frame_loss_percent 0.000
min_latency_ns 6760.000
max_latency_ns 19920.000
avg_latency_ns 8549.000
type rfc2544_throughput
packet_size 1518
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:27:36
start_time 2018-04-23 15:06:01
stop_time 2018-04-23 15:33:38
3.2.4.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.04
%system 0.02
%guest 0.00
%CPU 200.08
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 5056776
RSS 43027
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6624
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.4.7. Anomalies

No anomalies were detected during the course of this test.

3.2.4.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.5. Test ID: PVVP_BACK2BACK

3.2.5.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvvp

  • vswitchperf: GIT tag: 7440717327dabe7ae37f99cb060bce43a7878096

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

  • VNF: QemuDpdkVhostUser, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvvp_back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: pvvp
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
3.2.5.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.5.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 425
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 64
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:26:08
start_time 2018-04-23 16:50:41
stop_time 2018-04-23 17:16:49
guest_loopback_app testpmd testpmd
3.2.5.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 600.04
%system 0.02
%guest 0.00
%CPU 600.18
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 17713420
RSS 43076
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6780
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 0.03
%guest 200.02
%CPU 200.08
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2850725
RSS 66567
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 14.60
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.02
%system 0.02
%guest 200.02
%CPU 200.06
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2871792
RSS 67968
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
3.2.5.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.5.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 483
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 128
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:26:08
start_time 2018-04-23 16:50:41
stop_time 2018-04-23 17:16:49
guest_loopback_app testpmd testpmd
3.2.5.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 600.04
%system 0.02
%guest 0.00
%CPU 600.18
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 17713420
RSS 43076
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6780
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 0.03
%guest 200.02
%CPU 200.08
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2850725
RSS 66567
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 14.60
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.02
%system 0.02
%guest 200.02
%CPU 200.06
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2871792
RSS 67968
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
3.2.5.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.5.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 70488721
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:26:08
start_time 2018-04-23 16:50:41
stop_time 2018-04-23 17:16:49
guest_loopback_app testpmd testpmd
3.2.5.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 600.04
%system 0.02
%guest 0.00
%CPU 600.18
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 17713420
RSS 43076
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6780
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 0.03
%guest 200.02
%CPU 200.08
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2850725
RSS 66567
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 14.60
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.02
%system 0.02
%guest 200.02
%CPU 200.06
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2871792
RSS 67968
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
3.2.5.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.5.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 35919540
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:26:08
start_time 2018-04-23 16:50:41
stop_time 2018-04-23 17:16:49
guest_loopback_app testpmd testpmd
3.2.5.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 600.04
%system 0.02
%guest 0.00
%CPU 600.18
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 17713420
RSS 43076
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6780
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 0.03
%guest 200.02
%CPU 200.08
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2850725
RSS 66567
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 14.60
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.02
%system 0.02
%guest 200.02
%CPU 200.06
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2871792
RSS 67968
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
3.2.5.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.5.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 24382314
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:26:08
start_time 2018-04-23 16:50:41
stop_time 2018-04-23 17:16:49
guest_loopback_app testpmd testpmd
3.2.5.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 600.04
%system 0.02
%guest 0.00
%CPU 600.18
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 17713420
RSS 43076
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6780
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 0.03
%guest 200.02
%CPU 200.08
CPU
minflt/s 0.02
majflt/s 0.00
VSZ 2850725
RSS 66567
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 14.60
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.02
%system 0.02
%guest 200.02
%CPU 200.06
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2871792
RSS 67968
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
3.2.5.7. Anomalies

No anomalies were detected during the course of this test.

3.2.5.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.6. Test ID: PVP_BACK2BACK

3.2.6.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: 7440717327dabe7ae37f99cb060bce43a7878096

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

  • VNF: QemuDpdkVhostUser, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvp_back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: pvp
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
3.2.6.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.6.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 2128
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 64
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:25:30
start_time 2018-04-23 16:07:03
stop_time 2018-04-23 16:32:34
guest_loopback_app testpmd
3.2.6.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.04
%system 0.02
%guest 0.00
%CPU 400.18
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 11421956
RSS 43068
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6788
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.04
%system 0.04
%guest 200.02
%CPU 200.11
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2810938
RSS 66947
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 14.32
kB_ccwr/s 0.00
3.2.6.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.6.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1449
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 128
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:25:30
start_time 2018-04-23 16:07:03
stop_time 2018-04-23 16:32:34
guest_loopback_app testpmd
3.2.6.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.04
%system 0.02
%guest 0.00
%CPU 400.18
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 11421956
RSS 43068
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6788
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.04
%system 0.04
%guest 200.02
%CPU 200.11
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2810938
RSS 66947
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 14.32
kB_ccwr/s 0.00
3.2.6.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.6.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 70488721
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:25:30
start_time 2018-04-23 16:07:03
stop_time 2018-04-23 16:32:34
guest_loopback_app testpmd
3.2.6.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.04
%system 0.02
%guest 0.00
%CPU 400.18
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 11421956
RSS 43068
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6788
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.04
%system 0.04
%guest 200.02
%CPU 200.11
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2810938
RSS 66947
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 14.32
kB_ccwr/s 0.00
3.2.6.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.6.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 35919540
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:25:30
start_time 2018-04-23 16:07:03
stop_time 2018-04-23 16:32:34
guest_loopback_app testpmd
3.2.6.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.04
%system 0.02
%guest 0.00
%CPU 400.18
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 11421956
RSS 43068
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6788
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.04
%system 0.04
%guest 200.02
%CPU 200.11
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2810938
RSS 66947
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 14.32
kB_ccwr/s 0.00
3.2.6.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.6.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 24382314
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:25:30
start_time 2018-04-23 16:07:03
stop_time 2018-04-23 16:32:34
guest_loopback_app testpmd
3.2.6.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.04
%system 0.02
%guest 0.00
%CPU 400.18
CPU
minflt/s 0.04
majflt/s 0.00
VSZ 11421956
RSS 43068
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6788
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.04
%system 0.04
%guest 200.02
%CPU 200.11
CPU
minflt/s 0.01
majflt/s 0.00
VSZ 2810938
RSS 66947
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 14.32
kB_ccwr/s 0.00
3.2.6.7. Anomalies

No anomalies were detected during the course of this test.

3.2.6.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.7. Test ID: PHY2PHY_TPUT

3.2.7.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 7440717327dabe7ae37f99cb060bce43a7878096

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: phy2phy_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: p2p
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
3.2.7.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.7.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 23316600.112
tx_rate_mbps Unknown
throughput_rx_mbps 11938.099
tx_rate_percent 78.344
throughput_rx_percent 78.344
frame_loss_percent 0.000
min_latency_ns 4580.000
max_latency_ns 39820.000
avg_latency_ns 11591.500
type rfc2544_throughput
packet_size 64
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:13:07
start_time 2018-04-23 14:22:59
stop_time 2018-04-23 14:36:06
3.2.7.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.04
%system 0.02
%guest 0.00
%CPU 200.16
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 5056768
RSS 42673
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6872
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.7.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.7.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 16891716.560
tx_rate_mbps Unknown
throughput_rx_mbps 17297.118
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 13200.000
max_latency_ns 45820.000
avg_latency_ns 34884.000
type rfc2544_throughput
packet_size 128
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:13:07
start_time 2018-04-23 14:22:59
stop_time 2018-04-23 14:36:06
3.2.7.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.04
%system 0.02
%guest 0.00
%CPU 200.16
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 5056768
RSS 42673
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6872
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.7.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.7.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699202.816
tx_rate_mbps Unknown
throughput_rx_mbps 19247.935
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 13600.000
max_latency_ns 45640.000
avg_latency_ns 33661.500
type rfc2544_throughput
packet_size 512
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:13:07
start_time 2018-04-23 14:22:59
stop_time 2018-04-23 14:36:06
3.2.7.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.04
%system 0.02
%guest 0.00
%CPU 200.16
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 5056768
RSS 42673
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6872
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.7.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.7.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394616.743
tx_rate_mbps Unknown
throughput_rx_mbps 19616.700
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 13240.000
max_latency_ns 37600.000
avg_latency_ns 27548.000
type rfc2544_throughput
packet_size 1024
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:13:07
start_time 2018-04-23 14:22:59
stop_time 2018-04-23 14:36:06
3.2.7.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.04
%system 0.02
%guest 0.00
%CPU 200.16
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 5056768
RSS 42673
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6872
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.7.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.7.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625474.812
tx_rate_mbps Unknown
throughput_rx_mbps 19739.766
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 15160.000
max_latency_ns 39940.000
avg_latency_ns 29334.000
type rfc2544_throughput
packet_size 1518
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:13:07
start_time 2018-04-23 14:22:59
stop_time 2018-04-23 14:36:06
3.2.7.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 200.04
%system 0.02
%guest 0.00
%CPU 200.16
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 5056768
RSS 42673
%MEM 0.06
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6872
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
3.2.7.7. Anomalies

No anomalies were detected during the course of this test.

3.2.7.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.2.8. Test ID: PVP_TPUT

3.2.8.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: 7440717327dabe7ae37f99cb060bce43a7878096

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsDpdkVhost, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

  • VNF: QemuDpdkVhostUser, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvp_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvp
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
3.2.8.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

3.2.8.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 7663723.107
tx_rate_mbps Unknown
throughput_rx_mbps 3923.826
tx_rate_percent 25.750
throughput_rx_percent 25.750
frame_loss_percent 0.000
min_latency_ns 5880.000
max_latency_ns 39840.000
avg_latency_ns 7437.000
type rfc2544_throughput
packet_size 64
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:17:07
start_time 2018-04-23 15:49:56
stop_time 2018-04-23 16:07:03
guest_loopback_app testpmd
3.2.8.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.04
%system 0.02
%guest 0.00
%CPU 400.17
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 11421964
RSS 43108
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6784
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.25
%system 0.22
%guest 200.11
%CPU 200.59
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 2791833
RSS 67103
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 22.10
kB_ccwr/s 0.00
3.2.8.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

3.2.8.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 8530446.409
tx_rate_mbps Unknown
throughput_rx_mbps 8735.177
tx_rate_percent 50.500
throughput_rx_percent 50.500
frame_loss_percent 0.000
min_latency_ns 7640.000
max_latency_ns 39520.000
avg_latency_ns 9689.000
type rfc2544_throughput
packet_size 128
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:17:07
start_time 2018-04-23 15:49:56
stop_time 2018-04-23 16:07:03
guest_loopback_app testpmd
3.2.8.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.04
%system 0.02
%guest 0.00
%CPU 400.17
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 11421964
RSS 43108
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6784
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.25
%system 0.22
%guest 200.11
%CPU 200.59
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 2791833
RSS 67103
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 22.10
kB_ccwr/s 0.00
3.2.8.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

3.2.8.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699188.781
tx_rate_mbps Unknown
throughput_rx_mbps 19247.877
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 21900.000
max_latency_ns 60460.000
avg_latency_ns 49693.500
type rfc2544_throughput
packet_size 512
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:17:07
start_time 2018-04-23 15:49:56
stop_time 2018-04-23 16:07:03
guest_loopback_app testpmd
3.2.8.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.04
%system 0.02
%guest 0.00
%CPU 400.17
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 11421964
RSS 43108
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6784
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.25
%system 0.22
%guest 200.11
%CPU 200.59
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 2791833
RSS 67103
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 22.10
kB_ccwr/s 0.00
3.2.8.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

3.2.8.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394612.600
tx_rate_mbps Unknown
throughput_rx_mbps 19616.666
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 21420.000
max_latency_ns 51500.000
avg_latency_ns 40681.000
type rfc2544_throughput
packet_size 1024
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:17:07
start_time 2018-04-23 15:49:56
stop_time 2018-04-23 16:07:03
guest_loopback_app testpmd
3.2.8.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.04
%system 0.02
%guest 0.00
%CPU 400.17
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 11421964
RSS 43108
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6784
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.25
%system 0.22
%guest 200.11
%CPU 200.59
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 2791833
RSS 67103
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 22.10
kB_ccwr/s 0.00
3.2.8.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

3.2.8.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625472.265
tx_rate_mbps Unknown
throughput_rx_mbps 19739.735
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 22480.000
max_latency_ns 52320.000
avg_latency_ns 40900.500
type rfc2544_throughput
packet_size 1518
vswitch OvsDpdkVhost
traffic_type udp
test_execution_time 00:17:07
start_time 2018-04-23 15:49:56
stop_time 2018-04-23 16:07:03
guest_loopback_app testpmd
3.2.8.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 400.04
%system 0.02
%guest 0.00
%CPU 400.17
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 11421964
RSS 43108
%MEM 0.07
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.01
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 50660
RSS 6784
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.25
%system 0.22
%guest 200.11
%CPU 200.59
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 2791833
RSS 67103
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 22.10
kB_ccwr/s 0.00
3.2.8.7. Anomalies

No anomalies were detected during the course of this test.

3.2.8.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

3.3. Rationale for decisions

The tests conducted do not have pass/fail/conditional-pass criteria. The test is simply conducted and the results are reported.

3.4. Conclusions and recommendations

The test results are stable. The vsperf CI jobs that were used to obtain the results can be found at https://artifacts.opnfv.org/logs/vswitchperf/intel-pod12/2018-04-23_21-22-52/vswitchperf_logs_2018-04-23_21-22-52.tar.gz.

3.5. General

3.5.1. Glossary

  • NFV - Network Function Virtualization
  • Mbps - 1,000,000bps

3.5.2. Document change procedures and history

Document ID Author Date Modified
OPNFV_vswitchperf_LTR_ver_1.0_Jan_15_CN_DRAFT Christopher Nolan 23/01/2015
OPNFV_vswitchperf_LTR_ver_1.1_Jan_15_CN_DRAFT Christopher Nolan 28/01/2015

4. Performance report for OvsVanilla

4.1. Introduction

The objective of the OPNFV project titled “Characterise vSwitch Performance for Telco NFV Use Cases”, is to evaluate a virtual switch to identify its suitability for a Telco Network Function Virtualization (NFV) environment. As well as this, the project aims to identify any gaps or bottlenecks in order to drive architectural changes to improve virtual switch performance and determinism. The purpose of this document is to summarize the results of the tests carried out on the virtual switch in the Network Function Virtualization Infrastructure (NFVI) and, from these results, provide evaluations and recommendations for the virtual switch. Test results will be outlined in details-of-LTR, preceded by the document-identifier and the scope and references).

This document is currently in draft form.

4.1.1. Document identifier

The document id will be used to uniquely identify versions of the LTR. The format for the document id will be: OPNFV_vswitchperf_LTR_rel_STATUS, the status is one of: DRAFT, REVIEWED, CORRECTED or FINAL. The document id for this version of the LTR is: OPNFV_vswitchperf_LTR_Brahmaputra_DRAFT.

4.1.2. Scope

The scope of this report is to detail the results of the tests that have been performed on the virtual switch. This report will also evaluate the results of these tests and, based on these evaluations, provide recommendations on the suitability of the virtual switch for use in a Telco NFV environment.

4.2. Details of the Level Test Report

This section provides a test-results-overview. Also included are the rationale and the conclusions.

4.2.1. Test ID: BACK2BACK

4.2.1.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 7440717327dabe7ae37f99cb060bce43a7878096

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

Below are test details:

  • Test ID: back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: p2p
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
4.2.1.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.1.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 5108
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:18
start_time 2018-04-23 17:44:45
stop_time 2018-04-23 18:28:03
4.2.1.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.27
%system 1.17
%guest 0.00
%CPU 1.47
CPU
minflt/s 0.08
majflt/s 0.00
VSZ 3364068
RSS 7771
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5704
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.1.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 3382
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:18
start_time 2018-04-23 17:44:45
stop_time 2018-04-23 18:28:03
4.2.1.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.27
%system 1.17
%guest 0.00
%CPU 1.47
CPU
minflt/s 0.08
majflt/s 0.00
VSZ 3364068
RSS 7771
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5704
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.1.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1613
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:18
start_time 2018-04-23 17:44:45
stop_time 2018-04-23 18:28:03
4.2.1.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.27
%system 1.17
%guest 0.00
%CPU 1.47
CPU
minflt/s 0.08
majflt/s 0.00
VSZ 3364068
RSS 7771
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5704
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.1.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1370
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:18
start_time 2018-04-23 17:44:45
stop_time 2018-04-23 18:28:03
4.2.1.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.27
%system 1.17
%guest 0.00
%CPU 1.47
CPU
minflt/s 0.08
majflt/s 0.00
VSZ 3364068
RSS 7771
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5704
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.1.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1488
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:18
start_time 2018-04-23 17:44:45
stop_time 2018-04-23 18:28:03
4.2.1.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.27
%system 1.17
%guest 0.00
%CPU 1.47
CPU
minflt/s 0.08
majflt/s 0.00
VSZ 3364068
RSS 7771
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.00
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5704
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.1.7. Anomalies

No anomalies were detected during the course of this test.

4.2.1.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.2. Test ID: PHY2PHY_SCALABILITY

4.2.2.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 7440717327dabe7ae37f99cb060bce43a7878096

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

Below are test details:

  • Test ID: phy2phy_scalability
  • Description: LTD.Scalability.Flows.RFC2544.0PacketLoss
  • Deployment: p2p
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
4.2.2.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.2.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 988188.270
tx_rate_mbps Unknown
throughput_rx_mbps 505.952
tx_rate_percent 3.320
throughput_rx_percent 3.320
frame_loss_percent 0.000
min_latency_ns 4020.000
max_latency_ns 268340.000
avg_latency_ns 30748.500
type rfc2544_throughput
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:34
start_time 2018-04-23 18:55:39
stop_time 2018-04-23 19:23:14
stream_count 8000
match_type L4
pre-installed_flows No
4.2.2.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.28
%system 1.23
%guest 0.00
%CPU 1.49
CPU
minflt/s 1.39
majflt/s 0.00
VSZ 3364064
RSS 9215
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 6036
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.2.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.2.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 952809.444
tx_rate_mbps Unknown
throughput_rx_mbps 975.677
tx_rate_percent 5.641
throughput_rx_percent 5.641
frame_loss_percent 0.000
min_latency_ns 4160.000
max_latency_ns 1608660.000
avg_latency_ns 29041.000
type rfc2544_throughput
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:34
start_time 2018-04-23 18:55:39
stop_time 2018-04-23 19:23:14
stream_count 8000
match_type L4
pre-installed_flows No
4.2.2.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.28
%system 1.23
%guest 0.00
%CPU 1.49
CPU
minflt/s 1.39
majflt/s 0.00
VSZ 3364064
RSS 9215
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 6036
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.2.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.2.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 846599.874
tx_rate_mbps Unknown
throughput_rx_mbps 3467.673
tx_rate_percent 18.016
throughput_rx_percent 18.016
frame_loss_percent 0.000
min_latency_ns 5140.000
max_latency_ns 468700.000
avg_latency_ns 29289.000
type rfc2544_throughput
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:34
start_time 2018-04-23 18:55:39
stop_time 2018-04-23 19:23:14
stream_count 8000
match_type L4
pre-installed_flows No
4.2.2.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.28
%system 1.23
%guest 0.00
%CPU 1.49
CPU
minflt/s 1.39
majflt/s 0.00
VSZ 3364064
RSS 9215
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 6036
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.2.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.2.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 857393.395
tx_rate_mbps Unknown
throughput_rx_mbps 7023.767
tx_rate_percent 35.805
throughput_rx_percent 35.805
frame_loss_percent 0.000
min_latency_ns 5760.000
max_latency_ns 238340.000
avg_latency_ns 31149.000
type rfc2544_throughput
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:34
start_time 2018-04-23 18:55:39
stop_time 2018-04-23 19:23:14
stream_count 8000
match_type L4
pre-installed_flows No
4.2.2.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.28
%system 1.23
%guest 0.00
%CPU 1.49
CPU
minflt/s 1.39
majflt/s 0.00
VSZ 3364064
RSS 9215
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 6036
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.2.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.2.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 871160.000
tx_rate_mbps Unknown
throughput_rx_mbps 10579.367
tx_rate_percent 53.594
throughput_rx_percent 53.594
frame_loss_percent 0.000
min_latency_ns 8100.000
max_latency_ns 570820.000
avg_latency_ns 57537.500
type rfc2544_throughput
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:34
start_time 2018-04-23 18:55:39
stop_time 2018-04-23 19:23:14
stream_count 8000
match_type L4
pre-installed_flows No
4.2.2.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.28
%system 1.23
%guest 0.00
%CPU 1.49
CPU
minflt/s 1.39
majflt/s 0.00
VSZ 3364064
RSS 9215
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 6036
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.2.7. Anomalies

No anomalies were detected during the course of this test.

4.2.2.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.3. Test ID: PVVP_TPUT

4.2.3.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvvp

  • vswitchperf: GIT tag: 7440717327dabe7ae37f99cb060bce43a7878096

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • VNF: QemuVirtioNet, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvvp_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvvp
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
4.2.3.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.3.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 261097.651
tx_rate_mbps Unknown
throughput_rx_mbps 133.682
tx_rate_percent 1.773
throughput_rx_percent 0.877
frame_loss_percent 50.531
min_latency_ns 45400.000
max_latency_ns 33528780.000
avg_latency_ns 4436886.000
type rfc2544_throughput
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:29:08
start_time 2018-04-23 20:34:51
stop_time 2018-04-23 21:04:00
guest_loopback_app testpmd testpmd
4.2.3.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.27
%system 1.02
%guest 0.00
%CPU 1.27
CPU
minflt/s 0.15
majflt/s 0.00
VSZ 3364192
RSS 8067
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5984
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 1.63
%guest 198.42
%CPU 200.09
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2770404
RSS 65320
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 12.09
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.01
%system 1.64
%guest 198.38
%CPU 200.04
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2801000
RSS 65628
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
4.2.3.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.3.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 299569.449
tx_rate_mbps Unknown
throughput_rx_mbps 306.759
tx_rate_percent 1.773
throughput_rx_percent 1.773
frame_loss_percent 0.000
min_latency_ns 19720.000
max_latency_ns 701400.000
avg_latency_ns 72326.000
type rfc2544_throughput
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:29:08
start_time 2018-04-23 20:34:51
stop_time 2018-04-23 21:04:00
guest_loopback_app testpmd testpmd
4.2.3.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.27
%system 1.02
%guest 0.00
%CPU 1.27
CPU
minflt/s 0.15
majflt/s 0.00
VSZ 3364192
RSS 8067
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5984
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 1.63
%guest 198.42
%CPU 200.09
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2770404
RSS 65320
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 12.09
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.01
%system 1.64
%guest 198.38
%CPU 200.04
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2801000
RSS 65628
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
4.2.3.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.3.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 192375.853
tx_rate_mbps Unknown
throughput_rx_mbps 787.971
tx_rate_percent 4.094
throughput_rx_percent 4.094
frame_loss_percent 0.000
min_latency_ns 24580.000
max_latency_ns 1286220.000
avg_latency_ns 79053.500
type rfc2544_throughput
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:29:08
start_time 2018-04-23 20:34:51
stop_time 2018-04-23 21:04:00
guest_loopback_app testpmd testpmd
4.2.3.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.27
%system 1.02
%guest 0.00
%CPU 1.27
CPU
minflt/s 0.15
majflt/s 0.00
VSZ 3364192
RSS 8067
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5984
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 1.63
%guest 198.42
%CPU 200.09
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2770404
RSS 65320
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 12.09
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.01
%system 1.64
%guest 198.38
%CPU 200.04
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2801000
RSS 65628
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
4.2.3.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.3.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 283242.636
tx_rate_mbps Unknown
throughput_rx_mbps 2320.324
tx_rate_percent 11.828
throughput_rx_percent 11.828
frame_loss_percent 0.000
min_latency_ns 29840.000
max_latency_ns 993860.000
avg_latency_ns 90783.000
type rfc2544_throughput
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:29:08
start_time 2018-04-23 20:34:51
stop_time 2018-04-23 21:04:00
guest_loopback_app testpmd testpmd
4.2.3.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.27
%system 1.02
%guest 0.00
%CPU 1.27
CPU
minflt/s 0.15
majflt/s 0.00
VSZ 3364192
RSS 8067
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5984
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 1.63
%guest 198.42
%CPU 200.09
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2770404
RSS 65320
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 12.09
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.01
%system 1.64
%guest 198.38
%CPU 200.04
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2801000
RSS 65628
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
4.2.3.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.3.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 267698.488
tx_rate_mbps Unknown
throughput_rx_mbps 3250.930
tx_rate_percent 16.469
throughput_rx_percent 16.469
frame_loss_percent 0.000
min_latency_ns 30480.000
max_latency_ns 730640.000
avg_latency_ns 90744.500
type rfc2544_throughput
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:29:08
start_time 2018-04-23 20:34:51
stop_time 2018-04-23 21:04:00
guest_loopback_app testpmd testpmd
4.2.3.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.27
%system 1.02
%guest 0.00
%CPU 1.27
CPU
minflt/s 0.15
majflt/s 0.00
VSZ 3364192
RSS 8067
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5984
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.03
%system 1.63
%guest 198.42
%CPU 200.09
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2770404
RSS 65320
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 12.09
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.01
%system 1.64
%guest 198.38
%CPU 200.04
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2801000
RSS 65628
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.03
kB_ccwr/s 0.00
4.2.3.7. Anomalies

No anomalies were detected during the course of this test.

4.2.3.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.4. Test ID: PHY2PHY_TPUT_MOD_VLAN

4.2.4.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 7440717327dabe7ae37f99cb060bce43a7878096

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

Below are test details:

  • Test ID: phy2phy_tput_mod_vlan
  • Description: LTD.Throughput.RFC2544.PacketLossRatioFrameModification
  • Deployment: p2p
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
4.2.4.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.4.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 988188.306
tx_rate_mbps Unknown
throughput_rx_mbps 537.574
tx_rate_percent 3.320
throughput_rx_percent 3.478
frame_loss_percent 0.000
min_latency_ns 4020.000
max_latency_ns 9595860.000
avg_latency_ns 134709.000
type rfc2544_throughput
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:35
start_time 2018-04-23 18:28:03
stop_time 2018-04-23 18:55:38
4.2.4.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.27
%system 1.20
%guest 0.00
%CPU 1.48
CPU
minflt/s 0.16
majflt/s 0.00
VSZ 3364068
RSS 7628
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44276
RSS 5680
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.4.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.4.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 952809.715
tx_rate_mbps Unknown
throughput_rx_mbps 1006.167
tx_rate_percent 5.641
throughput_rx_percent 5.793
frame_loss_percent 0.000
min_latency_ns 4180.000
max_latency_ns 485560.000
avg_latency_ns 28105.000
type rfc2544_throughput
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:35
start_time 2018-04-23 18:28:03
stop_time 2018-04-23 18:55:38
4.2.4.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.27
%system 1.20
%guest 0.00
%CPU 1.48
CPU
minflt/s 0.16
majflt/s 0.00
VSZ 3364068
RSS 7628
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44276
RSS 5680
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.4.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.4.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 810254.190
tx_rate_mbps Unknown
throughput_rx_mbps 3344.729
tx_rate_percent 17.242
throughput_rx_percent 17.372
frame_loss_percent 0.000
min_latency_ns 5000.000
max_latency_ns 259300.000
avg_latency_ns 29847.000
type rfc2544_throughput
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:35
start_time 2018-04-23 18:28:03
stop_time 2018-04-23 18:55:38
4.2.4.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.27
%system 1.20
%guest 0.00
%CPU 1.48
CPU
minflt/s 0.16
majflt/s 0.00
VSZ 3364068
RSS 7628
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44276
RSS 5680
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.4.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.4.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 857393.215
tx_rate_mbps Unknown
throughput_rx_mbps 7051.202
tx_rate_percent 35.805
throughput_rx_percent 35.942
frame_loss_percent 0.000
min_latency_ns 5780.000
max_latency_ns 461520.000
avg_latency_ns 36115.500
type rfc2544_throughput
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:35
start_time 2018-04-23 18:28:03
stop_time 2018-04-23 18:55:38
4.2.4.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.27
%system 1.20
%guest 0.00
%CPU 1.48
CPU
minflt/s 0.16
majflt/s 0.00
VSZ 3364068
RSS 7628
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44276
RSS 5680
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.4.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.4.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 871159.197
tx_rate_mbps Unknown
throughput_rx_mbps 10607.234
tx_rate_percent 53.594
throughput_rx_percent 53.733
frame_loss_percent 0.000
min_latency_ns 8360.000
max_latency_ns 1576380.000
avg_latency_ns 141862.000
type rfc2544_throughput
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:35
start_time 2018-04-23 18:28:03
stop_time 2018-04-23 18:55:38
4.2.4.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.27
%system 1.20
%guest 0.00
%CPU 1.48
CPU
minflt/s 0.16
majflt/s 0.00
VSZ 3364068
RSS 7628
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44276
RSS 5680
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.4.7. Anomalies

No anomalies were detected during the course of this test.

4.2.4.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.5. Test ID: PVVP_BACK2BACK

4.2.5.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvvp

  • vswitchperf: GIT tag: 7440717327dabe7ae37f99cb060bce43a7878096

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • VNF: QemuVirtioNet, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvvp_back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: pvvp
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
4.2.5.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.5.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1702
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:59
start_time 2018-04-23 21:04:00
stop_time 2018-04-23 21:48:00
guest_loopback_app testpmd testpmd
4.2.5.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.28
%system 1.14
%guest 0.00
%CPU 1.43
CPU
minflt/s 0.08
majflt/s 0.00
VSZ 3364192
RSS 7999
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 6000
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.02
%system 0.13
%guest 199.90
%CPU 200.05
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2822588
RSS 66740
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.26
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.01
%system 0.10
%guest 199.93
%CPU 200.04
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2819504
RSS 66276
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.02
kB_ccwr/s 0.00
4.2.5.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.5.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544_back2back
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:59
start_time 2018-04-23 21:04:00
stop_time 2018-04-23 21:48:00
guest_loopback_app testpmd testpmd
4.2.5.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.28
%system 1.14
%guest 0.00
%CPU 1.43
CPU
minflt/s 0.08
majflt/s 0.00
VSZ 3364192
RSS 7999
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 6000
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.02
%system 0.13
%guest 199.90
%CPU 200.05
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2822588
RSS 66740
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.26
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.01
%system 0.10
%guest 199.93
%CPU 200.04
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2819504
RSS 66276
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.02
kB_ccwr/s 0.00
4.2.5.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.5.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1075
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:59
start_time 2018-04-23 21:04:00
stop_time 2018-04-23 21:48:00
guest_loopback_app testpmd testpmd
4.2.5.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.28
%system 1.14
%guest 0.00
%CPU 1.43
CPU
minflt/s 0.08
majflt/s 0.00
VSZ 3364192
RSS 7999
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 6000
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.02
%system 0.13
%guest 199.90
%CPU 200.05
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2822588
RSS 66740
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.26
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.01
%system 0.10
%guest 199.93
%CPU 200.04
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2819504
RSS 66276
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.02
kB_ccwr/s 0.00
4.2.5.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.5.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 1096
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:59
start_time 2018-04-23 21:04:00
stop_time 2018-04-23 21:48:00
guest_loopback_app testpmd testpmd
4.2.5.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.28
%system 1.14
%guest 0.00
%CPU 1.43
CPU
minflt/s 0.08
majflt/s 0.00
VSZ 3364192
RSS 7999
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 6000
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.02
%system 0.13
%guest 199.90
%CPU 200.05
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2822588
RSS 66740
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.26
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.01
%system 0.10
%guest 199.93
%CPU 200.04
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2819504
RSS 66276
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.02
kB_ccwr/s 0.00
4.2.5.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.5.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 744
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:59
start_time 2018-04-23 21:04:00
stop_time 2018-04-23 21:48:00
guest_loopback_app testpmd testpmd
4.2.5.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.28
%system 1.14
%guest 0.00
%CPU 1.43
CPU
minflt/s 0.08
majflt/s 0.00
VSZ 3364192
RSS 7999
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 6000
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.02
%system 0.13
%guest 199.90
%CPU 200.05
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2822588
RSS 66740
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.26
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.01
%system 0.10
%guest 199.93
%CPU 200.04
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2819504
RSS 66276
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 0.02
kB_ccwr/s 0.00
4.2.5.7. Anomalies

No anomalies were detected during the course of this test.

4.2.5.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.6. Test ID: PVP_BACK2BACK

4.2.6.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: 7440717327dabe7ae37f99cb060bce43a7878096

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • VNF: QemuVirtioNet, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvp_back2back
  • Description: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: pvp
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
4.2.6.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.6.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 425
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:08
start_time 2018-04-23 19:51:42
stop_time 2018-04-23 20:34:50
guest_loopback_app testpmd
4.2.6.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.26
%system 1.14
%guest 0.00
%CPU 1.47
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 3364068
RSS 7763
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5832
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.02
%system 0.10
%guest 199.94
%CPU 200.05
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 2841307
RSS 66800
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.44
kB_ccwr/s 0.00
4.2.6.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.6.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544_back2back
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:08
start_time 2018-04-23 19:51:42
stop_time 2018-04-23 20:34:50
guest_loopback_app testpmd
4.2.6.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.26
%system 1.14
%guest 0.00
%CPU 1.47
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 3364068
RSS 7763
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5832
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.02
%system 0.10
%guest 199.94
%CPU 200.05
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 2841307
RSS 66800
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.44
kB_ccwr/s 0.00
4.2.6.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.6.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 537
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:08
start_time 2018-04-23 19:51:42
stop_time 2018-04-23 20:34:50
guest_loopback_app testpmd
4.2.6.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.26
%system 1.14
%guest 0.00
%CPU 1.47
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 3364068
RSS 7763
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5832
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.02
%system 0.10
%guest 199.94
%CPU 200.05
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 2841307
RSS 66800
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.44
kB_ccwr/s 0.00
4.2.6.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.6.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 548
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:08
start_time 2018-04-23 19:51:42
stop_time 2018-04-23 20:34:50
guest_loopback_app testpmd
4.2.6.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.26
%system 1.14
%guest 0.00
%CPU 1.47
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 3364068
RSS 7763
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5832
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.02
%system 0.10
%guest 199.94
%CPU 200.05
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 2841307
RSS 66800
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.44
kB_ccwr/s 0.00
4.2.6.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.6.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 372
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:43:08
start_time 2018-04-23 19:51:42
stop_time 2018-04-23 20:34:50
guest_loopback_app testpmd
4.2.6.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.26
%system 1.14
%guest 0.00
%CPU 1.47
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 3364068
RSS 7763
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5832
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.02
%system 0.10
%guest 199.94
%CPU 200.05
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 2841307
RSS 66800
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 8.44
kB_ccwr/s 0.00
4.2.6.7. Anomalies

No anomalies were detected during the course of this test.

4.2.6.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.7. Test ID: PHY2PHY_TPUT

4.2.7.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: p2p

  • vswitchperf: GIT tag: 7440717327dabe7ae37f99cb060bce43a7878096

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

Below are test details:

  • Test ID: phy2phy_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: p2p
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
4.2.7.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.7.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 988188.749
tx_rate_mbps Unknown
throughput_rx_mbps 505.953
tx_rate_percent 3.320
throughput_rx_percent 3.320
frame_loss_percent 0.000
min_latency_ns 4000.000
max_latency_ns 3754380.000
avg_latency_ns 35153.500
type rfc2544_throughput
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:46
start_time 2018-04-23 17:16:58
stop_time 2018-04-23 17:44:44
4.2.7.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.27
%system 1.16
%guest 0.00
%CPU 1.47
CPU
minflt/s 0.16
majflt/s 0.00
VSZ 3364068
RSS 7695
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5504
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.7.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.7.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 952809.174
tx_rate_mbps Unknown
throughput_rx_mbps 975.677
tx_rate_percent 5.641
throughput_rx_percent 5.641
frame_loss_percent 0.000
min_latency_ns 4140.000
max_latency_ns 202380.000
avg_latency_ns 28395.500
type rfc2544_throughput
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:46
start_time 2018-04-23 17:16:58
stop_time 2018-04-23 17:44:44
4.2.7.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.27
%system 1.16
%guest 0.00
%CPU 1.47
CPU
minflt/s 0.16
majflt/s 0.00
VSZ 3364068
RSS 7695
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5504
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.7.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.7.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 846599.908
tx_rate_mbps Unknown
throughput_rx_mbps 3467.673
tx_rate_percent 18.016
throughput_rx_percent 18.016
frame_loss_percent 0.000
min_latency_ns 5000.000
max_latency_ns 466720.000
avg_latency_ns 29551.500
type rfc2544_throughput
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:46
start_time 2018-04-23 17:16:58
stop_time 2018-04-23 17:44:44
4.2.7.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.27
%system 1.16
%guest 0.00
%CPU 1.47
CPU
minflt/s 0.16
majflt/s 0.00
VSZ 3364068
RSS 7695
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5504
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.7.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.7.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 875914.165
tx_rate_mbps Unknown
throughput_rx_mbps 7175.489
tx_rate_percent 36.578
throughput_rx_percent 36.578
frame_loss_percent 0.000
min_latency_ns 5780.000
max_latency_ns 421460.000
avg_latency_ns 36044.000
type rfc2544_throughput
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:46
start_time 2018-04-23 17:16:58
stop_time 2018-04-23 17:44:44
4.2.7.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.27
%system 1.16
%guest 0.00
%CPU 1.47
CPU
minflt/s 0.16
majflt/s 0.00
VSZ 3364068
RSS 7695
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5504
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.7.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.7.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 858588.781
tx_rate_mbps Unknown
throughput_rx_mbps 10426.702
tx_rate_percent 52.820
throughput_rx_percent 52.820
frame_loss_percent 0.000
min_latency_ns 8900.000
max_latency_ns 402920.000
avg_latency_ns 34889.500
type rfc2544_throughput
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:27:46
start_time 2018-04-23 17:16:58
stop_time 2018-04-23 17:44:44
4.2.7.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.27
%system 1.16
%guest 0.00
%CPU 1.47
CPU
minflt/s 0.16
majflt/s 0.00
VSZ 3364068
RSS 7695
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5504
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
4.2.7.7. Anomalies

No anomalies were detected during the course of this test.

4.2.7.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.2.8. Test ID: PVP_TPUT

4.2.8.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: 7440717327dabe7ae37f99cb060bce43a7878096

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: OvsVanilla, Version: 2.8.1, GIT tag: 5c67e9cfc1eaab19187720c857841be6c8046c40

  • VNF: QemuVirtioNet, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvp_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvp
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
4.2.8.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

4.2.8.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 282341.270
tx_rate_mbps Unknown
throughput_rx_mbps 144.559
tx_rate_percent 1.773
throughput_rx_percent 0.949
frame_loss_percent 46.507
min_latency_ns 10080.000
max_latency_ns 74660000.000
avg_latency_ns 3775789.000
type rfc2544_throughput
packet_size 64
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:28:26
start_time 2018-04-23 19:23:14
stop_time 2018-04-23 19:51:41
guest_loopback_app testpmd
4.2.8.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.28
%system 1.10
%guest 0.00
%CPU 1.37
CPU
minflt/s 0.18
majflt/s 0.00
VSZ 3364064
RSS 7607
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5976
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.02
%system 1.22
%guest 198.81
%CPU 200.05
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2836980
RSS 67271
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.38
kB_ccwr/s 0.00
4.2.8.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

4.2.8.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 299567.780
tx_rate_mbps Unknown
throughput_rx_mbps 306.757
tx_rate_percent 1.773
throughput_rx_percent 1.773
frame_loss_percent 0.000
min_latency_ns 11220.000
max_latency_ns 839020.000
avg_latency_ns 49853.000
type rfc2544_throughput
packet_size 128
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:28:26
start_time 2018-04-23 19:23:14
stop_time 2018-04-23 19:51:41
guest_loopback_app testpmd
4.2.8.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.28
%system 1.10
%guest 0.00
%CPU 1.37
CPU
minflt/s 0.18
majflt/s 0.00
VSZ 3364064
RSS 7607
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5976
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.02
%system 1.22
%guest 198.81
%CPU 200.05
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2836980
RSS 67271
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.38
kB_ccwr/s 0.00
4.2.8.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

4.2.8.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 265067.111
tx_rate_mbps Unknown
throughput_rx_mbps 1085.715
tx_rate_percent 5.641
throughput_rx_percent 5.641
frame_loss_percent 0.000
min_latency_ns 14860.000
max_latency_ns 649320.000
avg_latency_ns 68106.500
type rfc2544_throughput
packet_size 512
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:28:26
start_time 2018-04-23 19:23:14
stop_time 2018-04-23 19:51:41
guest_loopback_app testpmd
4.2.8.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.28
%system 1.10
%guest 0.00
%CPU 1.37
CPU
minflt/s 0.18
majflt/s 0.00
VSZ 3364064
RSS 7607
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5976
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.02
%system 1.22
%guest 198.81
%CPU 200.05
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2836980
RSS 67271
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.38
kB_ccwr/s 0.00
4.2.8.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

4.2.8.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 320281.612
tx_rate_mbps Unknown
throughput_rx_mbps 2623.747
tx_rate_percent 13.375
throughput_rx_percent 13.375
frame_loss_percent 0.000
min_latency_ns 20720.000
max_latency_ns 642740.000
avg_latency_ns 81021.000
type rfc2544_throughput
packet_size 1024
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:28:26
start_time 2018-04-23 19:23:14
stop_time 2018-04-23 19:51:41
guest_loopback_app testpmd
4.2.8.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.28
%system 1.10
%guest 0.00
%CPU 1.37
CPU
minflt/s 0.18
majflt/s 0.00
VSZ 3364064
RSS 7607
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5976
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.02
%system 1.22
%guest 198.81
%CPU 200.05
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2836980
RSS 67271
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.38
kB_ccwr/s 0.00
4.2.8.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

4.2.8.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 192264.965
tx_rate_mbps Unknown
throughput_rx_mbps 2334.866
tx_rate_percent 11.828
throughput_rx_percent 11.828
frame_loss_percent 0.000
min_latency_ns 16520.000
max_latency_ns 548200.000
avg_latency_ns 61287.000
type rfc2544_throughput
packet_size 1518
vswitch OvsVanilla
traffic_type udp
test_execution_time 00:28:26
start_time 2018-04-23 19:23:14
stop_time 2018-04-23 19:51:41
guest_loopback_app testpmd
4.2.8.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: ovs-vswitchd
Statistic Value
%usr 0.28
%system 1.10
%guest 0.00
%CPU 1.37
CPU
minflt/s 0.18
majflt/s 0.00
VSZ 3364064
RSS 7607
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.01
kB_ccwr/s 0.00
Process: ovsdb-server
Statistic Value
%usr 0.00
%system 0.00
%guest 0.00
%CPU 0.01
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 44280
RSS 5976
%MEM 0.01
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
Process: qemu-system-x86
Statistic Value
%usr 0.02
%system 1.22
%guest 198.81
%CPU 200.05
CPU
minflt/s 0.00
majflt/s 0.00
VSZ 2836980
RSS 67271
%MEM 0.10
kB_rd/s 0.00
kB_wr/s 13.38
kB_ccwr/s 0.00
4.2.8.7. Anomalies

No anomalies were detected during the course of this test.

4.2.8.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

4.3. Rationale for decisions

The tests conducted do not have pass/fail/conditional-pass criteria. The test is simply conducted and the results are reported.

4.4. Conclusions and recommendations

The test results are stable. The vsperf CI jobs that were used to obtain the results can be found at https://artifacts.opnfv.org/logs/vswitchperf/intel-pod12/2018-04-23_21-22-52/vswitchperf_logs_2018-04-23_21-22-52.tar.gz.

4.5. General

4.5.1. Glossary

  • NFV - Network Function Virtualization
  • Mbps - 1,000,000bps

4.5.2. Document change procedures and history

Document ID Author Date Modified
OPNFV_vswitchperf_LTR_ver_1.0_Jan_15_CN_DRAFT Christopher Nolan 23/01/2015
OPNFV_vswitchperf_LTR_ver_1.1_Jan_15_CN_DRAFT Christopher Nolan 28/01/2015

5. Performance report for VppDpdkVhost

5.1. Introduction

The objective of the OPNFV project titled “Characterise vSwitch Performance for Telco NFV Use Cases”, is to evaluate a virtual switch to identify its suitability for a Telco Network Function Virtualization (NFV) environment. As well as this, the project aims to identify any gaps or bottlenecks in order to drive architectural changes to improve virtual switch performance and determinism. The purpose of this document is to summarize the results of the tests carried out on the virtual switch in the Network Function Virtualization Infrastructure (NFVI) and, from these results, provide evaluations and recommendations for the virtual switch. Test results will be outlined in details-of-LTR, preceded by the document-identifier and the scope and references).

This document is currently in draft form.

5.1.1. Document identifier

The document id will be used to uniquely identify versions of the LTR. The format for the document id will be: OPNFV_vswitchperf_LTR_rel_STATUS, the status is one of: DRAFT, REVIEWED, CORRECTED or FINAL. The document id for this version of the LTR is: OPNFV_vswitchperf_LTR_Brahmaputra_DRAFT.

5.1.2. Scope

The scope of this report is to detail the results of the tests that have been performed on the virtual switch. This report will also evaluate the results of these tests and, based on these evaluations, provide recommendations on the suitability of the virtual switch for use in a Telco NFV environment.

5.2. Details of the Level Test Report

This section provides a test-results-overview. Also included are the rationale and the conclusions.

5.2.1. Test ID: PVP_TPUT_VPP

5.2.1.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: clean

  • vswitchperf: GIT tag: 7440717327dabe7ae37f99cb060bce43a7878096

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: VppDpdkVhost, Version: v17.07-release, GIT tag: NA

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvp_tput_vpp
  • Description: VPP: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: clean
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
5.2.1.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

5.2.1.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 5822177.247
tx_rate_mbps Unknown
throughput_rx_mbps 2980.955
tx_rate_percent 19.563
throughput_rx_percent 19.563
frame_loss_percent 0.000
min_latency_ns 9060.000
max_latency_ns 76580.000
avg_latency_ns 21823.000
type rfc2544_throughput
packet_size 64
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:23:35
start_time 2018-04-23 22:33:39
stop_time 2018-04-23 22:57:14
5.2.1.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.90
%system 0.29
%guest 0.00
%CPU 601.19
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 9749181
RSS 50950
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.1.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

5.2.1.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 6440002.185
tx_rate_mbps Unknown
throughput_rx_mbps 6594.562
tx_rate_percent 38.125
throughput_rx_percent 38.125
frame_loss_percent 0.000
min_latency_ns 9500.000
max_latency_ns 82580.000
avg_latency_ns 36364.500
type rfc2544_throughput
packet_size 128
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:23:35
start_time 2018-04-23 22:33:39
stop_time 2018-04-23 22:57:14
5.2.1.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.90
%system 0.29
%guest 0.00
%CPU 601.19
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 9749181
RSS 50950
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.1.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

5.2.1.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4590209.167
tx_rate_mbps Unknown
throughput_rx_mbps 18801.497
tx_rate_percent 97.680
throughput_rx_percent 97.680
frame_loss_percent 0.000
min_latency_ns 10620.000
max_latency_ns 113420.000
avg_latency_ns 59369.000
type rfc2544_throughput
packet_size 512
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:23:35
start_time 2018-04-23 22:33:39
stop_time 2018-04-23 22:57:14
5.2.1.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.90
%system 0.29
%guest 0.00
%CPU 601.19
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 9749181
RSS 50950
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.1.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

5.2.1.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394563.552
tx_rate_mbps Unknown
throughput_rx_mbps 19616.265
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 10340.000
max_latency_ns 106920.000
avg_latency_ns 91080.500
type rfc2544_throughput
packet_size 1024
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:23:35
start_time 2018-04-23 22:33:39
stop_time 2018-04-23 22:57:14
5.2.1.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.90
%system 0.29
%guest 0.00
%CPU 601.19
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 9749181
RSS 50950
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.1.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

5.2.1.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625449.986
tx_rate_mbps Unknown
throughput_rx_mbps 19739.465
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 9960.000
max_latency_ns 80420.000
avg_latency_ns 69511.000
type rfc2544_throughput
packet_size 1518
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:23:35
start_time 2018-04-23 22:33:39
stop_time 2018-04-23 22:57:14
5.2.1.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.90
%system 0.29
%guest 0.00
%CPU 601.19
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 9749181
RSS 50950
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.1.7. Anomalies

No anomalies were detected during the course of this test.

5.2.1.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

5.2.2. Test ID: PVP_BACK2BACK_VPP

5.2.2.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: clean

  • vswitchperf: GIT tag: 7440717327dabe7ae37f99cb060bce43a7878096

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: VppDpdkVhost, Version: v17.07-release, GIT tag: NA

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvp_back2back_vpp
  • Description: VPP: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: clean
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
5.2.2.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

5.2.2.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544_back2back
packet_size 64
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:34:56
start_time 2018-04-23 22:57:14
stop_time 2018-04-23 23:32:11
5.2.2.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.75
%system 0.29
%guest 0.00
%CPU 601.03
CPU
minflt/s 0.06
majflt/s 0.00
VSZ 9902955
RSS 51009
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.2.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

5.2.2.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544_back2back
packet_size 128
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:34:56
start_time 2018-04-23 22:57:14
stop_time 2018-04-23 23:32:11
5.2.2.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.75
%system 0.29
%guest 0.00
%CPU 601.03
CPU
minflt/s 0.06
majflt/s 0.00
VSZ 9902955
RSS 51009
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.2.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

5.2.2.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 70488657
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:34:56
start_time 2018-04-23 22:57:14
stop_time 2018-04-23 23:32:11
5.2.2.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.75
%system 0.29
%guest 0.00
%CPU 601.03
CPU
minflt/s 0.06
majflt/s 0.00
VSZ 9902955
RSS 51009
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.2.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

5.2.2.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 35919540
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:34:56
start_time 2018-04-23 22:57:14
stop_time 2018-04-23 23:32:11
5.2.2.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.75
%system 0.29
%guest 0.00
%CPU 601.03
CPU
minflt/s 0.06
majflt/s 0.00
VSZ 9902955
RSS 51009
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.2.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

5.2.2.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 24382314
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:34:56
start_time 2018-04-23 22:57:14
stop_time 2018-04-23 23:32:11
5.2.2.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.75
%system 0.29
%guest 0.00
%CPU 601.03
CPU
minflt/s 0.06
majflt/s 0.00
VSZ 9902955
RSS 51009
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.2.7. Anomalies

No anomalies were detected during the course of this test.

5.2.2.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

5.2.3. Test ID: PVVP_BACK2BACK_VPP

5.2.3.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: clean

  • vswitchperf: GIT tag: 7440717327dabe7ae37f99cb060bce43a7878096

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: VppDpdkVhost, Version: v17.07-release, GIT tag: NA

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvvp_back2back_vpp
  • Description: VPP: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: clean
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
5.2.3.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

5.2.3.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544_back2back
packet_size 64
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:36:42
start_time 2018-04-24 00:04:33
stop_time 2018-04-24 00:41:15
5.2.3.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.99
%system 0.30
%guest 0.00
%CPU 601.30
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 15488646
RSS 51241
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.3.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

5.2.3.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 0
b2b_frame_loss_percent 100
type rfc2544_back2back
packet_size 128
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:36:42
start_time 2018-04-24 00:04:33
stop_time 2018-04-24 00:41:15
5.2.3.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.99
%system 0.30
%guest 0.00
%CPU 601.30
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 15488646
RSS 51241
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.3.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

5.2.3.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 70488721
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:36:42
start_time 2018-04-24 00:04:33
stop_time 2018-04-24 00:41:15
5.2.3.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.99
%system 0.30
%guest 0.00
%CPU 601.30
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 15488646
RSS 51241
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.3.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

5.2.3.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 35919540
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:36:42
start_time 2018-04-24 00:04:33
stop_time 2018-04-24 00:41:15
5.2.3.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.99
%system 0.30
%guest 0.00
%CPU 601.30
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 15488646
RSS 51241
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.3.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

5.2.3.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 24382314
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:36:42
start_time 2018-04-24 00:04:33
stop_time 2018-04-24 00:41:15
5.2.3.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.99
%system 0.30
%guest 0.00
%CPU 601.30
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 15488646
RSS 51241
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.3.7. Anomalies

No anomalies were detected during the course of this test.

5.2.3.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

5.2.4. Test ID: PHY2PHY_BACK2BACK_VPP

5.2.4.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: clean

  • vswitchperf: GIT tag: 7440717327dabe7ae37f99cb060bce43a7878096

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: VppDpdkVhost, Version: v17.07-release, GIT tag: NA

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: phy2phy_back2back_vpp
  • Description: VPP: LTD.Throughput.RFC2544.BackToBackFrames
  • Deployment: clean
  • Traffic type: rfc2544_back2back
  • Bidirectional : True
5.2.4.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

5.2.4.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 24267
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 64
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:26:55
start_time 2018-04-23 22:06:43
stop_time 2018-04-23 22:33:38
5.2.4.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.63
%system 0.28
%guest 0.00
%CPU 600.91
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 3937544
RSS 50488
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.4.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

5.2.4.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 262904
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 128
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:26:55
start_time 2018-04-23 22:06:43
stop_time 2018-04-23 22:33:38
5.2.4.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.63
%system 0.28
%guest 0.00
%CPU 600.91
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 3937544
RSS 50488
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.4.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

5.2.4.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 70488721
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 512
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:26:55
start_time 2018-04-23 22:06:43
stop_time 2018-04-23 22:33:38
5.2.4.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.63
%system 0.28
%guest 0.00
%CPU 600.91
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 3937544
RSS 50488
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.4.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

5.2.4.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 35919540
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1024
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:26:55
start_time 2018-04-23 22:06:43
stop_time 2018-04-23 22:33:38
5.2.4.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.63
%system 0.28
%guest 0.00
%CPU 600.91
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 3937544
RSS 50488
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.4.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

5.2.4.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
b2b_frames 24382314
b2b_frame_loss_percent 0.0
type rfc2544_back2back
packet_size 1518
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:26:55
start_time 2018-04-23 22:06:43
stop_time 2018-04-23 22:33:38
5.2.4.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.63
%system 0.28
%guest 0.00
%CPU 600.91
CPU
minflt/s 0.03
majflt/s 0.00
VSZ 3937544
RSS 50488
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.4.7. Anomalies

No anomalies were detected during the course of this test.

5.2.4.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

5.2.5. Test ID: PVVP_TPUT_VPP

5.2.5.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: clean

  • vswitchperf: GIT tag: 7440717327dabe7ae37f99cb060bce43a7878096

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: VppDpdkVhost, Version: None, GIT tag: None

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvvp_tput_vpp
  • Description: VPP: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: clean
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
5.2.5.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

5.2.5.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 527809.064
tx_rate_mbps Unknown
throughput_rx_mbps 270.238
tx_rate_percent 1.773
throughput_rx_percent 1.773
frame_loss_percent 0.000
min_latency_ns 6240.000
max_latency_ns 450540.000
avg_latency_ns 19312.500
type rfc2544_throughput
packet_size 64
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:32:21
start_time 2018-04-23 23:32:11
stop_time 2018-04-24 00:04:32
5.2.5.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 601.22
%system 0.68
%guest 0.00
%CPU 601.90
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 15359358
RSS 51147
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.5.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

5.2.5.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 560863.883
tx_rate_mbps Unknown
throughput_rx_mbps 574.325
tx_rate_percent 3.320
throughput_rx_percent 3.320
frame_loss_percent 0.000
min_latency_ns 6500.000
max_latency_ns 481860.000
avg_latency_ns 19791.500
type rfc2544_throughput
packet_size 128
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:32:21
start_time 2018-04-23 23:32:11
stop_time 2018-04-24 00:04:32
5.2.5.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 601.22
%system 0.68
%guest 0.00
%CPU 601.90
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 15359358
RSS 51147
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.5.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

5.2.5.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 664870.498
tx_rate_mbps Unknown
throughput_rx_mbps 2723.310
tx_rate_percent 14.148
throughput_rx_percent 14.148
frame_loss_percent 0.000
min_latency_ns 7280.000
max_latency_ns 456960.000
avg_latency_ns 21549.500
type rfc2544_throughput
packet_size 512
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:32:21
start_time 2018-04-23 23:32:11
stop_time 2018-04-24 00:04:32
5.2.5.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 601.22
%system 0.68
%guest 0.00
%CPU 601.90
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 15359358
RSS 51147
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.5.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

5.2.5.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2264985.520
tx_rate_mbps Unknown
throughput_rx_mbps 18554.761
tx_rate_percent 94.586
throughput_rx_percent 94.586
frame_loss_percent 0.000
min_latency_ns 28360.000
max_latency_ns 348400.000
avg_latency_ns 72040.000
type rfc2544_throughput
packet_size 1024
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:32:21
start_time 2018-04-23 23:32:11
stop_time 2018-04-24 00:04:32
5.2.5.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 601.22
%system 0.68
%guest 0.00
%CPU 601.90
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 15359358
RSS 51147
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.5.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

5.2.5.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1474619.727
tx_rate_mbps Unknown
throughput_rx_mbps 17907.782
tx_rate_percent 90.719
throughput_rx_percent 90.719
frame_loss_percent 0.000
min_latency_ns 21360.000
max_latency_ns 202400.000
avg_latency_ns 60517.000
type rfc2544_throughput
packet_size 1518
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:32:21
start_time 2018-04-23 23:32:11
stop_time 2018-04-24 00:04:32
5.2.5.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 601.22
%system 0.68
%guest 0.00
%CPU 601.90
CPU
minflt/s 0.09
majflt/s 0.00
VSZ 15359358
RSS 51147
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.5.7. Anomalies

No anomalies were detected during the course of this test.

5.2.5.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

5.2.6. Test ID: PHY2PHY_TPUT_VPP

5.2.6.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
    • Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: clean

  • vswitchperf: GIT tag: 7440717327dabe7ae37f99cb060bce43a7878096

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: VppDpdkVhost, Version: v17.07-release, GIT tag: NA

  • DPDK Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: phy2phy_tput_vpp
  • Description: VPP: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: clean
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
5.2.6.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

5.2.6.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 22856315.450
tx_rate_mbps Unknown
throughput_rx_mbps 11702.434
tx_rate_percent 76.797
throughput_rx_percent 76.797
frame_loss_percent 0.000
min_latency_ns 5420.000
max_latency_ns 62980.000
avg_latency_ns 22129.000
type rfc2544_throughput
packet_size 64
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:18:31
start_time 2018-04-23 21:48:11
stop_time 2018-04-23 22:06:43
5.2.6.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.71
%system 0.31
%guest 0.00
%CPU 601.01
CPU
minflt/s 0.06
majflt/s 0.00
VSZ 3937544
RSS 50568
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.6.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

5.2.6.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 16630608.311
tx_rate_mbps Unknown
throughput_rx_mbps 17029.743
tx_rate_percent 98.453
throughput_rx_percent 98.453
frame_loss_percent 0.000
min_latency_ns 5760.000
max_latency_ns 55520.000
avg_latency_ns 12818.500
type rfc2544_throughput
packet_size 128
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:18:31
start_time 2018-04-23 21:48:11
stop_time 2018-04-23 22:06:43
5.2.6.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.71
%system 0.31
%guest 0.00
%CPU 601.01
CPU
minflt/s 0.06
majflt/s 0.00
VSZ 3937544
RSS 50568
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.6.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

5.2.6.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699211.745
tx_rate_mbps Unknown
throughput_rx_mbps 19247.971
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 5920.000
max_latency_ns 30020.000
avg_latency_ns 19032.500
type rfc2544_throughput
packet_size 512
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:18:31
start_time 2018-04-23 21:48:11
stop_time 2018-04-23 22:06:43
5.2.6.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.71
%system 0.31
%guest 0.00
%CPU 601.01
CPU
minflt/s 0.06
majflt/s 0.00
VSZ 3937544
RSS 50568
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.6.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

5.2.6.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394617.893
tx_rate_mbps Unknown
throughput_rx_mbps 19616.710
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 6240.000
max_latency_ns 29840.000
avg_latency_ns 18938.500
type rfc2544_throughput
packet_size 1024
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:18:31
start_time 2018-04-23 21:48:11
stop_time 2018-04-23 22:06:43
5.2.6.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.71
%system 0.31
%guest 0.00
%CPU 601.01
CPU
minflt/s 0.06
majflt/s 0.00
VSZ 3937544
RSS 50568
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.6.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

5.2.6.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625474.719
tx_rate_mbps Unknown
throughput_rx_mbps 19739.765
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 6700.000
max_latency_ns 31920.000
avg_latency_ns 20442.000
type rfc2544_throughput
packet_size 1518
vswitch VppDpdkVhost
traffic_type udp
test_execution_time 00:18:31
start_time 2018-04-23 21:48:11
stop_time 2018-04-23 22:06:43
5.2.6.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: vpp_main
Statistic Value
%usr 600.71
%system 0.31
%guest 0.00
%CPU 601.01
CPU
minflt/s 0.06
majflt/s 0.00
VSZ 3937544
RSS 50568
%MEM 0.08
kB_rd/s 0.00
kB_wr/s 0.00
kB_ccwr/s 0.00
5.2.6.7. Anomalies

No anomalies were detected during the course of this test.

5.2.6.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

5.3. Rationale for decisions

The tests conducted do not have pass/fail/conditional-pass criteria. The test is simply conducted and the results are reported.

5.4. Conclusions and recommendations

The test results are stable. The vsperf CI jobs that were used to obtain the results can be found at https://artifacts.opnfv.org/logs/vswitchperf/intel-pod12/2018-04-23_21-22-52/vswitchperf_logs_2018-04-23_21-22-52.tar.gz.

5.5. General

5.5.1. Glossary

  • NFV - Network Function Virtualization
  • Mbps - 1,000,000bps

5.5.2. Document change procedures and history

Document ID Author Date Modified
OPNFV_vswitchperf_LTR_ver_1.0_Jan_15_CN_DRAFT Christopher Nolan 23/01/2015
OPNFV_vswitchperf_LTR_ver_1.1_Jan_15_CN_DRAFT Christopher Nolan 28/01/2015

6. Performance report for none

6.1. Introduction

The objective of the OPNFV project titled “Characterise vSwitch Performance for Telco NFV Use Cases”, is to evaluate a virtual switch to identify its suitability for a Telco Network Function Virtualization (NFV) environment. As well as this, the project aims to identify any gaps or bottlenecks in order to drive architectural changes to improve virtual switch performance and determinism. The purpose of this document is to summarize the results of the tests carried out on the virtual switch in the Network Function Virtualization Infrastructure (NFVI) and, from these results, provide evaluations and recommendations for the virtual switch. Test results will be outlined in details-of-LTR, preceded by the document-identifier and the scope and references).

This document is currently in draft form.

6.1.1. Document identifier

The document id will be used to uniquely identify versions of the LTR. The format for the document id will be: OPNFV_vswitchperf_LTR_rel_STATUS, the status is one of: DRAFT, REVIEWED, CORRECTED or FINAL. The document id for this version of the LTR is: OPNFV_vswitchperf_LTR_Brahmaputra_DRAFT.

6.1.2. Scope

The scope of this report is to detail the results of the tests that have been performed on the virtual switch. This report will also evaluate the results of these tests and, based on these evaluations, provide recommendations on the suitability of the virtual switch for use in a Telco NFV environment.

6.2. Details of the Level Test Report

This section provides a test-results-overview. Also included are the rationale and the conclusions.

6.2.1. Test ID: PVP_TPUT

6.2.1.1. Test Environment

Below is the environment that the test was performed in:

  • OS: CentOS Linux 7 Core

  • Kernel Version: 4.4.112-1.el7.elrepo.x86_64

  • NIC(s):
    • Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
    • Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)

  • Board: Intel Corporation S2600WT2R [2 sockets]

  • CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

  • CPU cores: 44

  • Memory: 65917716 kB

  • Virtual Switch Set-up: pvp

  • vswitchperf: GIT tag: 7440717327dabe7ae37f99cb060bce43a7878096

  • Traffic Generator: IxNet, Version: 8.01.1029.6, GIT tag: None

  • vSwitch: none, Version: None, GIT tag: None

  • VNF: QemuPciPassthrough, Version: 2.9.1, GIT tag: 4cd42653f5c1df326a2678a84f24a78fb9601277

  • VM images:
    • /home/jenkins/vloop-vnf-ubuntu-14.04_20160823.qcow2
  • VM loopback apps:
    • testpmd, Version: 17.08.0, GIT tag: 02657b4adcb8af773e26ec061b01cd7abdd3f0b6

Below are test details:

  • Test ID: pvp_tput
  • Description: LTD.Throughput.RFC2544.PacketLossRatio
  • Deployment: pvp
  • Traffic type: rfc2544_throughput
  • Bidirectional : True
6.2.1.2. Test results for packet size: 64

A detailed summary of the main results is outlined below.

6.2.1.2.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4210844.929
tx_rate_mbps Unknown
throughput_rx_mbps 2155.953
tx_rate_percent 14.148
throughput_rx_percent 14.148
frame_loss_percent 0.000
min_latency_ns 3440.000
max_latency_ns 43440.000
avg_latency_ns 3718.500
type rfc2544_throughput
packet_size 64
vswitch none
traffic_type udp
test_execution_time 00:19:22
start_time 2018-04-24 00:41:27
stop_time 2018-04-24 01:00:49
guest_loopback_app testpmd
6.2.1.2.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.02
%system 0.04
%guest 200.02
%CPU 200.09
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 2878059
RSS 76715
%MEM 0.12
kB_rd/s 0.00
kB_wr/s 19.29
kB_ccwr/s 0.00
6.2.1.3. Test results for packet size: 128

A detailed summary of the main results is outlined below.

6.2.1.3.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 5264207.719
tx_rate_mbps Unknown
throughput_rx_mbps 5390.549
tx_rate_percent 31.164
throughput_rx_percent 31.164
frame_loss_percent 0.000
min_latency_ns 3660.000
max_latency_ns 39620.000
avg_latency_ns 4081.500
type rfc2544_throughput
packet_size 128
vswitch none
traffic_type udp
test_execution_time 00:19:22
start_time 2018-04-24 00:41:27
stop_time 2018-04-24 01:00:49
guest_loopback_app testpmd
6.2.1.3.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.02
%system 0.04
%guest 200.02
%CPU 200.09
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 2878059
RSS 76715
%MEM 0.12
kB_rd/s 0.00
kB_wr/s 19.29
kB_ccwr/s 0.00
6.2.1.4. Test results for packet size: 512

A detailed summary of the main results is outlined below.

6.2.1.4.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 4699179.540
tx_rate_mbps Unknown
throughput_rx_mbps 19247.839
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 4840.000
max_latency_ns 48760.000
avg_latency_ns 34325.000
type rfc2544_throughput
packet_size 512
vswitch none
traffic_type udp
test_execution_time 00:19:22
start_time 2018-04-24 00:41:27
stop_time 2018-04-24 01:00:49
guest_loopback_app testpmd
6.2.1.4.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.02
%system 0.04
%guest 200.02
%CPU 200.09
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 2878059
RSS 76715
%MEM 0.12
kB_rd/s 0.00
kB_wr/s 19.29
kB_ccwr/s 0.00
6.2.1.5. Test results for packet size: 1024

A detailed summary of the main results is outlined below.

6.2.1.5.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 2394604.180
tx_rate_mbps Unknown
throughput_rx_mbps 19616.597
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 6780.000
max_latency_ns 52340.000
avg_latency_ns 34147.500
type rfc2544_throughput
packet_size 1024
vswitch none
traffic_type udp
test_execution_time 00:19:22
start_time 2018-04-24 00:41:27
stop_time 2018-04-24 01:00:49
guest_loopback_app testpmd
6.2.1.5.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.02
%system 0.04
%guest 200.02
%CPU 200.09
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 2878059
RSS 76715
%MEM 0.12
kB_rd/s 0.00
kB_wr/s 19.29
kB_ccwr/s 0.00
6.2.1.6. Test results for packet size: 1518

A detailed summary of the main results is outlined below.

6.2.1.6.1. Results/Metrics Collected

The following are the metrics obtained during this test:

Metric Result
tx_rate_fps Unknown
throughput_rx_fps 1625462.431
tx_rate_mbps Unknown
throughput_rx_mbps 19739.616
tx_rate_percent 100
throughput_rx_percent 100
frame_loss_percent 0.000
min_latency_ns 6400.000
max_latency_ns 59040.000
avg_latency_ns 39800.000
type rfc2544_throughput
packet_size 1518
vswitch none
traffic_type udp
test_execution_time 00:19:22
start_time 2018-04-24 00:41:27
stop_time 2018-04-24 01:00:49
guest_loopback_app testpmd
6.2.1.6.2. Statistics collected

The following system statistics were collected during testcase execution:

Process: qemu-system-x86
Statistic Value
%usr 0.02
%system 0.04
%guest 200.02
%CPU 200.09
CPU
minflt/s 0.05
majflt/s 0.00
VSZ 2878059
RSS 76715
%MEM 0.12
kB_rd/s 0.00
kB_wr/s 19.29
kB_ccwr/s 0.00
6.2.1.7. Anomalies

No anomalies were detected during the course of this test.

6.2.1.8. Testing Activities/Events

pidstat is used to collect the process statistics, as such some values such as %CPU and %USER maybe > 100% as the values are summed across multiple cores. For more info on pidstat please see: http://linux.die.net/man/1/pidstat. Please note that vsperf recalculates the CPU consumption of a process by aggregating the CPU usage of each thread.

Known issues: Some reported metrics have the value “unkown”. These values are marked unknown as they are not values retrieved from the external tester (traffic generator). They were incorrectly derived in a way that made assumptions about packet sizes, as such they have been deprecated from vsperf and marked as unknown. They will be resolved in the next release.

6.3. Rationale for decisions

The tests conducted do not have pass/fail/conditional-pass criteria. The test is simply conducted and the results are reported.

6.4. Conclusions and recommendations

The test results are stable. The vsperf CI jobs that were used to obtain the results can be found at https://artifacts.opnfv.org/logs/vswitchperf/intel-pod12/2018-04-23_21-22-52/vswitchperf_logs_2018-04-23_21-22-52.tar.gz.

6.5. General

6.5.1. Glossary

  • NFV - Network Function Virtualization
  • Mbps - 1,000,000bps

6.5.2. Document change procedures and history

Document ID Author Date Modified
OPNFV_vswitchperf_LTR_ver_1.0_Jan_15_CN_DRAFT Christopher Nolan 23/01/2015
OPNFV_vswitchperf_LTR_ver_1.1_Jan_15_CN_DRAFT Christopher Nolan 28/01/2015