Sample Performance Benchmarks for IBM MQ Service Virtualization

The ldt development team performed testing to show benchmarks for performance of service virtualization of IBM MQ. You can use these benchmarks to help tune the performance of your system.
dts101
The DevTest development team performed testing to show benchmarks for performance of service virtualization of IBM MQ. You can use these benchmarks to help tune the performance of your system.
Environment Details
Payload
Size
XML Elements
100B
4
2 kB
75
10 kB
116
100 kB
2802
Machine Settings
Settings
 
CPU
Intel Xeon E5-2680 2.5 GHz
Virtual cores
16
Memory
16 GB
OS
Windows Server 2012 R2
MQ Server
8.0.0.5
CA Service Virtualization Version
10.1
  • DevTest Settings
    • lisa.pathfinder.on=false
      This option disables the feature Pathfinder. (also called as Application Insight). If you are not using this feature, it is better to turn it off to improve performance.
    • lisa.vse.performance.enabled=true
      If your licensing terms allow, you can turn on this option to scale your Virtual Services. This option would allow increasing the capacity of virtual services to more than one. For normal use of a virtual service, the default capacity of one is sufficient. For more information, see Deploy Virtual Services.
    • vse.socket.reuse=true
      This setting improves efficiency of virtual services by reusing SOCKET objects. Without this option, a new socket and connection will be established for every request, which increases the response time and reduces throughput significantly.
    • lisa.net.timeout.ms=60000
      The timeout value (in milliseconds) used by the underlying messaging system. The messaging system is the way various components of DevTest Solutions communicate with each other.
    • lisa.net.xstream=false
      This setting is used by the internal messaging system and allows a messaging format of 'OBJECT'.
    • activemq.msgCompression=true
      This is an internal communication system related setting that compresses messages to speed up the communication among components and help them with higher throughput.
    • lisa.dcm.maxExpandPerLoadChange=50
      Obsolete setting. Do not use.
    • lisa.jpm.MaxInstances=5000
      UNIT:COUNT -This setting determines the maximum number of jobs that the system can handle. The jobs are test instances, virtual service listening threads, and units of work.
    • lisa.jpm.unlimited=false
      This setting ensures that the system does not keep taking unlimited load (jobs). This setting is required for the maxInstances setting to take effect.
    • lisa.overloadThreshold=30000
      UNIT: MILLISECONDS -The TestNode attempts to figure out if the Simulator is thrashing by checking the actual amount of time slept in think time vs. the amount of think time it was supposed to take. This setting is the amount of additional "slip" in think time that is acceptable before sending a warning TestEvent that the Simulator is overloaded.
    • lisa.eventPool.maxQueueSize=65535
      UNIT:COUNT - This determines the number of events that the simulator can queue up for processing.
    • lisa.eclipselink.cache.timeout.ms=3600000
      UNIT:MILLISECONDS - This setting determines the lifetime of objects in the persistence layer
    • lisa.CycleExecHistory.buffer.size=2
      UNIT:COUNT - Number of cycle event data that is kept in memory before being flushed out.
    • lisa.vse.metrics.sample.interval=1m
      UNIT:MINUTES - This property controls how often transaction rate and response times are sampled.
  •  
    JVM Settings (vmoptions)
    • -Xmx2048m
      Max HEAP memory that the JVM is entitled to use. You may tweak this setting to a higher value based on your scenario
    • -XX:+UseConcMarkSweepGC
      Specifies the type of Garbage collector that the JVM should use. This GC type suits applications that prefer shorter garbage collection pauses.
    • -Dorg.apache.xml.dtm.DTMManager=org.apache.xml.dtm.ref.DTMManagerDefault
      This is optimization to improve performance for the Generic XML Payload Parser.
       
  •  
    Scope of MQ request/response queues is set to 
    model.
     
     
  • Concurrent capacity increased to 30.
  • Think time set to 0.
Test Results
Payload
Number of transactions in VSI
Data Protocol Filter
SV [T/s]
CPU %
for the SV machine
Memory %
for the SV machine
100 B
300
Generic XML Payload Parser
6500 T/s
75%
70%
3 kB
300
Generic XML Payload Parser
5500 T/s
80%
70%
10 kB
300
Generic XML Payload Parser
5000 T/s
80%
70%
100 kB
300
Generic XML Payload Parser
1200 T/s
80%
70%
100 B
3000
Generic XML Payload Parser
5500 T/s
80%
70%
3 kB
3000
Generic XML Payload Parser
5300 T/s
80%
70%
10 kB
3000
Generic XML Payload Parser
5000 T/s
80%
70%
100 kB
3000
Generic XML Payload Parser
1100 T/s
80%
70%
100B
3000
XML Data Protocol
5600 T/s
99%
65%
The Generic XML Payload Parser always selects one XML element as a key.
Test Results Summary
The deprecated IBM MQSeries transport protocol has poor performance and should not be used in any performance tests. Use the IBM MQ Native (with queues defined as assets) instead.
Performance depends significantly on the number of arguments that are used for requests comparison. So the Generic XML Payload is better for larger payloads if a limited set of arguments is used for response selection. XML Data Protocol is not so efficient if the payload is large, as it automatically includes every XML element as an argument for matching. This results in a much larger set of match criteria, which takes longer to evaluate against incoming requests.
This applies for every protocol where these DPHs are used.