Performance testing of multi-user software systems is typically performed by emulating activities of multiple users working with the system simultaneously. These virtual users collect performance measures like response time and correctness while loading the system during the test. During a typical load scenario number of simultaneous virtual users gradually increases from one to some intended maximum, allowing to find out statistical relation between number of simultaneous users and performance measures.
However having fixed load scenario has several drawbacks. Usually the test has to be repeated iteratively with adjusted load scenarios, taking into account results acquired from previous test iterations. As load scenarios have to be adjusted manually and repetitions of the test may take considerable time, this approach seems to be ineffective.
This paper presents an alternative approach, where load scenario is adjusted automatically during the test, taking into account performance measures already gathered in the same test. Drawbacks of fixed load scenarios are analysed and methods of eliminating them by automatic adjustment are described. A case study showing implementation of the method and results obtained by applying it to performance testing of a real system is provided.
If the inline PDF is not rendering correctly, you can download the PDF file here.
G. Denaro A. Polini and W. Emmerich "Early performance testing of distributed software applications" in 4th International Workshop on Software and Performance (WOSP '04) January 2004. USA: ACM 2004 pp. 94-103.
D. P. Olshefski J. Nieh and D. Agrawal "Inferring client response time at the web server" in International Conference on Measurements and Modeling of Computer Systems (SIGMETRICS 2002) June 2002. USA: ACM 2002 pp. 160-171.
B. M. Subraya Integrated Approach to Web Performance Testing: A Practitioner's Guide. USA: IRM Press 2006.
C. Amza et al. "Specification and implementation of dynamic web site benchmarks" in 5th IEEE Workshop on Workload Characterization (WWC-5) November 2002. USA: IEEE Press 2002 pp 3-13.
M. J. Johnson et al. "Incorporating performance testing in test-driven development" IEEE Software vol. 24 no. 3. USA: IEEE Computer Society 2007 pp. 67-73.
D. Draheim et al "Realistic load testing of web applications" in 10th European Conference on Software Maintenance and Reengineering (CSMR 2006) March 2006. USA: IEEE Computer Society 2006 pp. 57-70.
A. Sukhorukov "Performance testing tool Picus" in 22nd International Conference on Systems for Automation of Engineering and Research (SAER-2008) September 2008. Bulgaria: King 2008 pp. 165-172.