Web load testing (article 2) - be generous with pacing

This is the second article explaining a practical approach to
avoiding misunderstandings concerning concurrent user load.

If you read the first article, you will be aware of my obsession with achieving even pacing for each test user. Here I will explain how to design tests to achieve even pacing.

First, a digression on thinking time and system time.

Most load testing tools allow the user thinking time to be simulated. In fact, most allow thinking time to be recorded during a user session so it can be replayed during the load test. I always include thinking time in the web session. This ensures that the total duration of each user session roughly corresponds with reality. This is important where the web system maintains sessions for users – you want the number of concurrent sessions to be realistic.

I also like to use transaction timers to measure ALL activity that is not user thinking time. In other words, I use timers to measure all system activity. Not surprisingly, I call this “system time”.

I estimate the minimum system time by running the web test script for a small number of users and adding up the times of all transactions. I multiply the system time by three to allow for slow-down under load, and add the thinking time. Then I add a comfortable margin for error, and end up with a pacing time that should be achieved– even under heavy load.

So there is my tip – allow plenty of time for each user session. Be generous with your pacing. I do everything I can to ensure there is a pause between user sessions so pacing is maintained.

The astute amongst my readers will have worked out that being generous with pacing implies a larger number of test users. Let me explain. To achieve a specified load (say 720 user sessions started per hour) with a pacing of 5 minutes requires 60 test users, whereas with a pacing of 10 minutes requires 120 test users. You should take this into account during test planning.