QTIP User Guide¶
QTIP CLI Usage¶
QTIP consists of a number of benchmarking tools or metrics, grouped under QPI’s. QPI’s map to the different components of a NFVI ecosystem, such as compute, network and storage. Depending on the type of application, a user may group them under plans.
QTIP CLI provides interface to all of the above the components. A help page provides a list of all the commands along with a short description.
qtip [-h|--help]
Typically a complete plan is executed at the target environment. QTIP defaults to a number of sample plans. One may be able to list them using
qtip plan list
One can also be able to view the details about a specific plan.
qtip plan show <plan_name>
where plan_name is one of those listed from the previous command.
To execute a complete plan
qtip plan run <plan_name>
Similarly, the same commands can be used for the other two components making up the plans, i.e QPI’s and metrics.
Debug option helps identify the error by providing a detailed traceback. It can be enabled as
qtip [-d|--debug] plan run <plan_name>
Compute Suite¶
Introduction¶
The QTIP testing suite aims to benchmark the compute components of an OPNFV platform. Such components include, the CPU performance, the memory performance. Additionally virtual computing performance provided by the Hypervisor (KVM) installed as part of OPNFV platforms would be benchmarked too.
The test suite consists of both synthetic and application specific benchmarks to test compute components.
All the compute benchmarks could be run in 2 scenarios:
- On Baremetal Machines provisioned by an OPNFV installer (Host machines)
- On Virtual Machines brought up through OpenStack on an OPNFV platform
Note: The Compute benchmank suite constains relatively old benchmarks such as dhrystone and whetstone. The suite would be updated for better benchmarks such as Linbench for the OPNFV C release.
Benchmarks¶
The benchmarks include:
Dhrystone 2.1¶
Dhrystone is a synthetic benchmark for measuring CPU performance. It uses integer calculations to evaluate CPU capabilities. Both Single CPU performance is measured along multi-cpu performance.
Dhrystone, however, is a dated benchmark and has some short comings. Written in C, it is a small program that doesn’t test the CPU memory subsystem. Additionally, dhrystone results could be modified by optimizing the compiler and insome cases hardware configuration.
References: http://www.eembc.org/techlit/datasheets/dhrystone_wp.pdf
Whetstone¶
Whetstone is a synthetic benchmark to measure CPU floating point operation performance. Both Single CPU performance is measured along multi-cpu performance.
Like Dhrystone, Whetstone is a dated benchmark and has short comings.
References:
OpenSSL Speed¶
OpenSSL Speed can be used to benchmark compute performance of a machine. In QTIP, two OpenSSL Speed benchmarks are incorporated: 1. RSA signatunes/sec signed by a machine 2. AES 128-bit encryption throughput for a machine for cipher block sizes
References:
RAMSpeed¶
RAMSpeed is used to measure a machine’s memory perfomace. The problem(array)size is large enough to ensure Cache Misses so that the main machine memory is used. INTmem and FLOATmem benchmarks are executed in 4 different scenarios:
- Copy: a(i)=b(i)
- Add: a(i)=b(i)+c(i)
- Scale: a(i)=b(i)*d
- Tniad: a(i)=b(i)+c(i)*d
INTmem uses integers in these four benchmarks whereas FLOATmem uses floating points for these benchmarks.
References:
DPI¶
nDPI is a modified variant of OpenDPI, Open source Deep packet Inspection, that is maintained by ntop. An example application called pcapreader has been developed and is available for use along nDPI.
A sample .pcap file is passed to the pcapreader application. nDPI classifies traffic in the pcap file into different categories based on string matching. The pcapreader application provides a throughput number for the rate at which traffic was classified, indicating a machine’s computational performance. The results are run 10 times and an average is taken for the obtained number.
nDPI may provide non consistent results and was added to Brahmaputra for experimental purposes
References:
http://www.ntop.org/products/deep-packet-inspection/ndpi/
http://www.ntop.org/wp-content/uploads/2013/12/nDPI_QuickStartGuide.pdf
