Sample Performance Benchmarks for Basic Service Virtualization

The DevTest development team performed testing to show benchmarks for performance of service virtualization. You can use these benchmarks to help tune the performance of your system.
dts101
The DevTest development team performed testing to show benchmarks for performance of service virtualization. You can use these benchmarks to help tune the performance of your system.
The following data represents a simple scenario to identify the behavior of
VSE
under heavy load. The variation of the independent variable has been limited to the payload size. Performance data has been collected for the following payload sizes: 1 KB, 10 KB, and 100 KB.
Environment Details
The following graphic pictures the topology of the Performance Lab.
Performance Lab Topology
System Specifications for All Machines Except JMeter
OS
Windows 2012 R2 64-bit VMs
vCPU
4 individual cores
assigned
and
reserved
Memory
16 GB
reserved
Hard Drive
100 GB
Processor
Intel® Xeon® CPU ES-2680 v3 @ 2.5GHz, 1 Core
DevTest
version:
10.1.0.108
JVM Arguments
  • -Xmx2048m
  • -XX:+UseConcMarkSweepGC
DevTest
Properties
  • lisa.vse.performance.enabled=true
  • lisa.pathfinder.on=false
  • vse.socket.reuse=true
  • lisa.net.timeout.ms=60000
  • lisa.net.xstream=false
  • lisa.dcm.maxExpandPerLoadChange=50
  • lisa.jpm.MaxInstances=5000
  • lisa.jpm.unlimited=false
  • lisa.overloadThreshold=30000
  • lisa.eventPool.maxQueueSize=65535
  • lisa.eclipselink.cache.timeout.ms=3600000
  • lisa.CycleExecHistory.buffer.size=2
Virtual Service configuration
Execution Mode
Most Efficient
Think Scale
0
Capacity
20
Transaction type
Stateless
Match Style
Signature
Log Level
VSE
: OFF
Other components
: WARN
A single virtual service was deployed to the
VSE
. Each HTTP call produces a single transaction count in the
VSE
. There is no search and replacement activity involved in this transaction. The service returns an exact match response. Further processing is required neither at the requests nor the response level.
Test Results
JSON Response Payload Size
Average TPS
Max TPS
Total Transactions per Run
Memory Utilization
(VSE Server)
CPU Utilization
(VSE Server)
Data Protocol
1 KB
14,806
16,886
53,301,355
18%
87%
REST Data Protocol
10 KB
10,887
12,770
39,193,259
19%
86%
REST Data Protocol
100 KB
4,336
4,830
15,608,915
19%
81%
REST Data Protocol
Duration of each test run:
60 minutes during off-peak hours
The data that are provided in the previous table represents the number of transactions that are handled by the
VSE
during each 60-minute test run.
System Resources Utilization
The following charts represent data that were captured from the
VSE
server machine during the execution of each test run. The machines hosting the database utilized a very modest amount of resources. The machine hosting the registry consumed limited CPU time (2 to 3%) and approximately 40% of the available RAM. Disk activity of the registry ranged from 50 to 100 KB/sec. This demonstrates that the machines supporting the
VSE
machine do not represent a bottleneck during the test runs.
Virtual Service Environments (VSEs) have a fixed limit of processing ten (10) transactions per second (TPS) by default. This limit is per VSE, not per virtual service. However, a properly sized VSE with "Performance Mode" enabled can handle over 11,500 TPS as the table above shows. To enable Performance Mode, you must contact Sales and purchase a "VSE for Performance" license for each performance VSE you wish to deploy, and you must contact Technical Support to enable Performance Mode on each performance VSE you purchase.
Legend
  • RED:  CPU Utilization
  • GREEN: Memory Available in GB (out of 16 GB)
Test Run With Payload of 1 KB
Test run with payload of 1 KB
Available Memory:
13.5 GB out of total of 16 GB
Test Run With Payload of 10 KB
Test run with payload of 10 KB
Available Memory:
13.4 GB out of total of 16 GB
Test Run With Payload of 100 KB
Test run with payload of 100 KB