Cloud Testing and Last Mile Testing in the real world

Cloud Testing. It's being talked about everywhere. But what does it all mean for performance testing in the real world ?

For some while now, there have been broadly two approaches to performance testing:

1) From the Internet. This is particularly suited to public Internet sites because all the components involved are tested - including your Internet connection and firewalls. For this to work effectively, load injection must be done from servers connected to the Internet at points which provide sufficient bandwidth. Broadly speaking, Cloud Testing takes this approach.

2) Within a private network. This is particularly suited for testing internal systems as load can be delivered from various points in your network to gather realistic performance results (and these can be geographically distant points on your network).

Over the years, numerous organisations have hosted remote testing services via the Internet, in some cases supplying a tool geared for customers to use, in other cases providing a complete test service. HP promote their Software-as-a-Service (SaaS) offering based around the well established Performance Center test tool. SOASTA have recently entered the market with their CloudTest web-load testing tool. This allows customers to quickly scale to high loads as the tool comes ready configured for deployment on various commercial Cloud services. Finally fans of JMeter are now gathering around a Cloud-based service called BlazeMeter.

Another test tool which advocates a slightly different approach is Gomez - now part of Compuware. Their philosophy is that Cloud Testing as described above does not provide sufficiently accurate end-to-end response times in the real world. Unless you can configure load injectors on the Internet in very narrowly defined geographic regions, you can't measure response times in those regions and performance will always be uncertain. And guess what - Gomez provides these highly distributed points of presence on the Internet, and has packaged this up as what they call Last Mile Testing. Take a look at their publications on this and see how well they match your performance testing issues. But if you just want to test your firewall, load balancer and so on - you may decide it does not matter too much where you test from.

There are no right or wrong answers for Cloud Testing - or Last Mile Testing. You can't escape from reviewing your business situation and defining specific requirements, assessing the risks if requirements are not met, and the costs of various testing approaches. So proceed with your eyes open - and be prepared to discuss the above areas in some detail along the way. And if you feel you need help along the way - do seek professional advice.