Determining the right timeout for your script can be tricky. The best practice is to use a time that when reached, your service or scenario under testing is to be considered fatally underperforming. For example, it is taking so long that the average user would give up trying.
Elements Delaying the Script
Some elements of the script make the script run longer, and likely to hit the timeout. You can remove these elements from the script without making any functional difference. Recording the script using the JMeter HTTP(s) Test Script Recording feature tends to be adding some of these elements automatically. If your script keeps hitting timeouts and includes these elements, remove or disable them unless they are important for your functional test scenario.
All of the "Timer" group of elements affect the script. The following is the list of JMeter 4.0 elements:
- Constant Timer
- Uniform Random Timer
- Precise Throughput Timer
- Constant Throughput Timer
- Gaussian Random Timer
- JSR223 Timer
- Poisson Random Timer
- Synchronizing Timer
- BeanShell Timer
Timeouts and Metrics
Unfortunately, all timeouts can result in incomplete or skewed metrics. ASM JMeter agent starts running the test script and two “wall-clock” timers at the same time. Once the first timer hits the timeout, the agent asks to stop running the script. This is called
graceful timeout. If it still does not stop when the second timer expires, the script is
aborted forcefully. The graceful shutdown initiated by the expiration of the first-timer gives JMeter some chance to finish requests in-process and finish writing the JTL file used for metric analysis. The forceful abort usually results in stopping all requests immediately and can lead to incomplete or even corrupt JTL files. This happens because some requests that completed before the timeout was hit are not flushed to the JTL file entirely and correctly. As the JTL file is incomplete or even corrupted, no precise metrics can then be extracted from it.
ASM JMeter agents try to minimize these chances but this is largely dependent on the JMeter itself and also on the script structure.
Types of Timeouts
The timeouts are categorized as follows:
- 7011 Script Run Timed-Out (total time metrics)This type of timeout will occur when the JMeter would close the running script gracefully by the first timer and the metrics inside the JTL file indicate that it was running a bit longer than the timeout specified. JTL file is usually not corrupt and metrics are not skewed much - they show a sum of what was executed and flushed to the JTL file even though it was only a part of the script.
- 1042 Script Run Timed-Out (measured actual run time)This type of timeout happens when the graceful-stop does not work and the script run had to be killed abruptly. Hence the metrics should not be trusted JTL file might be corrupted.
- Connection TimeoutsThis is not an error you see in the monitor log but they can be seen inside the JTL and also in the check details JTL overview in ASM UI. The error will be manifested as having a “Non HTTP response message: Connection timed out”. This signals that the connection was not even established because of some network problem getting in its way, most probably it’s being firewalled out somewhere. Why is this important? ASM JMeter agent can control this timeout only in limited ways, some aspects of it are mandated by the underlying OS. A script’s run can be pushed into constant timeout by just one step having this issue. If you’re seeing this constantly, the offending script step should be reviewed and made working or removed if possible.