Testing

What is Exploratory Testing?

In scripted testing you follow a script that tells you where to click, what to enter, and what to expect. Freestyle exploratory testing does not rely on scripts, but rather on the experience and intelligence of the tester. As you go you change your plan and approach based on what you have seen so far, just as you would when exploring an island or a new city.

Exploratory testing is not ‘just clicking around randomly’. It is quite the opposite. Good exploratory testing demands critical thinking, an eye for detail, a good deal of imagination and a natural curiosity for “what if”’s. It takes expertise, intuition and practice.

Exploratory testing is perhaps the most difficult and skilled form of manual testing. So I’ve gathered a few of my favourite resources to help you on your way.

In this blog, I want to share my experiences in organising a bug hunt. Testers at my organisation formed teams, who then set about testing a piece of the software that my own team had developed. It was a great learning experience for everyone involved, and something I’d highly recommend.

What is a Bug Hunt?

During a bug hunt, a Test Owner presents two teams (of two or more people) with a piece of software to be tested. The Test Owner provides some basic information, and the teams get to work testing the product. At the end of the session, they report their findings back to the Test Owner. The activity can be seen as training in how to organise and communicate testing, but is also a fun way to learn a new piece of software.

How Do You Organise a Bug Hunt?

Here are a few insights from my first experience as Test Owner. One of the hardest things for me was estimating how complex to make the test object. The most enriching on the other hand was seeing other testers dissect software that I had helped build. Because the teams were distributed, I also picked up tips on how to make working remotely go smoother.

Your experience will undoubtedly be a personal one. Nevertheless, the tips below will make the first time much easier.

How to test whether your API can handle anomalous HTTP responses.

Today I was exploratory testing my Service, which performs sequential HTTP requests that depend on the response of the previous request. I wanted to find out how our API would handle a variety of HTTP responses. I also wanted to see what would happen if things went wrong. What if the response contains an image? Or an error code? Or what if there was a timeout?

To answer these questions as quickly and easily as possible I ventured on to the web. With some effort, I found what I was looking for, and on the way I discovered a few tools that are just great for exploratory testing your API’s response handling. If you test or debug API’s on a regular basis, here are a few free tools you’ll definitely want to check out.

TestNet Spring Conference: Trends in Testing

A week or two ago I went to the TestNet 2017 Spring Event. I’m not going to recount the content of each talk or workshop I attended. Instead I want to combine this event with my experience at other conferences, and give you an overview of the biggest trends in agile testing right now.

This image perfectly pictures my first feelings about the conference, but let me go into a bit more detail first.
I signed up for the Google Testing blog more than a year ago. There I found a lot of interesting and useful reading about the world of automated testing. When I later got an email from Google informing me that there would be a conference held by them, I was not entirely sure wether I should apply. Is it relevant for me? Am I experienced enough to contribute? Well, what’s the worst that could happen? So I applied for it and did not regret it in the end.

What is a Root Cause Analysis?

Think for a minute. What usually happens when a serious bug makes it to production?

Someone walks by and asks you to look into it. ‘Find the cause’, they say. So you go in search of the cause, with the goal of fixing it as soon as possible. This is good, but it is by far not the most ideal situation. Let me explain.

Consider the very real error that led to the immediate destruction of an aircraft before take-off. In an unfortunate turn of events, instead of flipping the switch for raising the flaps, a pilot accidentally flipped the switch for raising the landing gear. Although nobody was injured the pilot’s mistake inconvenienced customers and led to expensive repairs. The cause of the failure was noted as pilot error. Why wasn’t this the end of it?

Reporting a bug right away after finding it might not be the most useful thing you can do. What.. how? Here is an example and the theory behind it.

Looking at the bug itself, not only what it can cause

My team refactored a part of our application to start using javascript. I was curious how it would react to the biggest numbers that can be typed in when I found a bug. There was a shopping cart involved and multiplying really big numbers got the cart frozen, it seemed. I pressed F12 to see what was going on behind the scenes. It wasn’t only because of big numbers, but instead it was a digit overflow, caused by a precision error in javascript (which I then quickly learned about: http://www.w3schools.com/js/js_numbers.asp) Aha! From then on I could easily reproduce it. I wondered, how else this could be a problem? This lead to me adding some pretty average-looking numbers which then would cause the same problem, locking down the cart.