Last week we published a new major version of the Visual Studio Performance Testing Quick Reference Guide. The effort of creating this document was lead by Geoff Gray. Geoff is a senior consultant in the Microsoft Services Labs in the US that specializes in performance testing using Visual Studio. I was part of the Rangers team that contributed articles to this guide.
From time to time I do performance testing engagements. Last week for example I worked for a customer and load tested their new Internet facing website before it goes live later this month. Performance testing uncovers issues in the software and the infrastructure that might otherwise go unnoticed until the public hits the site. And that is a bad time to discover performance issues.
Creating and executing real-world performance testing can be a tricky job and this quick reference guide gives lots of tricks & tips from consultants from Microsoft Services and the Visual Studio product team on how to tackle challenges you might encounter.
Visual Studio has a rich extensibility model for performance testing and that’s why it is interesting for development minded people like me. You can create plug-ins in C# to do advanced parameterization of web tests and load tests (example on MSDN). The QRG has lots of examples of when and how this is useful.
Load testing is part of Visual Studio 2008 Team Suite or Test Edition and of Visual Studio 2010 Ultimate. With VS2010, if you need to simulate more than 1000 users, you can purchase “virtual user packs”. This will enable you to simulate higher loads using a test rig consisting of one test controller and several test agents. Previously with VS2008 you had to license each agent. You can’t be sure in advance how many users you can simulate with one agent because it depends on many factors like hardware and complexity of the application and the test itself.
Many organizations forego the effort of performance testing and take a “wait and see” stance. This reactive approach can lead to costly repairs and bad publicity. My advice is to take a proactive stance and be confident about the performance of your application before it goes live. In many cases there are quick wins to improve the performance by enabling better caching (blob caching for SharePoint). Even simple things like enabling HTTP compression are still often forgotten.