Testing Services provides a platform for QA professionals to discuss and gain insights in to the business value delivered by testing, the best practices and processes that drive it and the emergence of new technologies that will shape the future of this profession.

Is the environment spoiling your party?

I frequently come across performance
testing projects entangled in cost and time overruns - the culprit usually
being environment issues. Since we can't wish away the environment and the
issues that come with it, the next best thing is to be better prepared by figuring
out the pitfalls and addressing them proactively.

Since a stable test environment is
critical for test script development, load simulation and bottleneck analysis
in the performance testing life cycle stages, let's take a good look at what to
watch out for when we prepare for a testing cycle.

Know thy application: Any environment issue during the test
execution phase is like the proverbial spanner in the works. It should come as
no surprise if the performance testing team has to spend significant effort in
debugging and analyzing it and, of course, following up with support teams for
resolution. To be effective in test environment risk assessment and issue resolution,
we must well know the application architecture, functionalities, workflows, and
the interconnecting components. Stubs and virtualization techniques can be
handy during test execution when one is familiar with the component level
details and how to use them. While investigating environment issues, the
development and infrastructure teams often seek the testing team's input - so lending
a hand with specifics will mean a faster turnaround.

Dependencies - wheels within wheels: Another party-pooper can be
the dependencies that may impact testing way before we even run the performance
test. Multi-tiered enterprise computing systems have these dependencies on each
tier and layer, within and outside the enterprise boundaries. In addition to the
functionalities, there are other factors at play that may impact the
performance test results. These could include high resource consumption by
another process hosted on the same infrastructure, execution of batch jobs, or parallel
test runs by another team. An outage in the environment during the test run can
force you to reschedule the test. That's why, it is all important to gather
information about all the possible dependencies that may impact the test
execution during the planning stage itself. It is always good to document these
issues as one comes across them for reference in future test cycles.

Stay in touch: The performance testing team cannot operate in a
vacuum. Team members must establish proper communication with the development,
infrastructure, functional and integration testing, and release management teams
right from the strategy phase to synchronize test preparation as well as
execution activities. The test schedule should be published well in advance in
case you are using a shared test environment. Notifications prior to running a
test must be sent out to the teams concerned to bring up the servers, mount
monitors, clear logs and keep the environment stable during the test execution.
A calendar that blocks shared computing resources can keep all stakeholders posted
on the date with the test execution and reduce retest efforts significantly. A
small tip: Keep your contact information handy and up-to-date.

Think ahead: Being well-prepared is half the battle won. For
testing, this means to think about possible environment failures and look for
workarounds well before the actual test execution. While preparing test
estimates, don't forget to factor in unknown environment issues that could adversely
impact the effort and the schedule. Keeping some buffers as a percentage of the
overall estimate can save you a lot of grief later. It is also important to
prioritize critical business transactions for the performance test so that, if
some functionalities flop during the planned test window, a test run on a
subset of the transactions can provide meaningful insights into application
performance. Finally, remember that time is your most precious resource. So, if
the test environment becomes completely unavailable during the planned test
window, utilize that time effectively in activities such as offline reporting
or knowledge management and promptly schedule the test in the next window available.

So these are a few best practices to
keep in mind while preparing for a performance test cycle. You can use these to
build your own set of rules specific to the challenges and constraints of your
set-up. The bottom line for a successful testing cycle is to keep tabs on
incidents and work through issues smartly.