Brothers in Arms, Part 1

I’m pretty sure it’s not exactly the best approach around to start your life in the blogosphere (or anywhere, for that matter) by criticizing people, but hey, this blog is about protesting against poor software quality and quality always starts with the people involved (feel free to contradict me on that, I’d love to see someone explain to me why I’m wrong) – their attitudes included – so if a big part of the problem is in the attitudes and mental approaches of the people instead of the methods or techniques used, then that’s what I will be protesting against today.

Starting the series…

As stated before, I will be posting a lot about my thoughts and views on what my colleagues in the testing world write in their own blogs and webpages or what I’ve been discussing with them through one medium or another.

The “honor” for the first post in this series goes to Rob Lambert for his excellent post Don’t be a follower, be a tester, which woke a multitude of emotions and thoughts in me. Actually, the post irritated me. Not because I would disagree, but because he’s so damn right in what he says. Briefly put: the irritating essence of the post is that the testing world is full of sheep bleating trendy catchphrases that make things sound really nice while actually meaning absolutely nothing (a favorite subject of criticism by Richard Rorty).

What really caught my attention were these three conclusions Rob made, based on the comments to one of his earlier posts:

We should not challenge the best practices of testing
We should not challenge the experts in testing
We should not talk about testing publicly
unless we are an expert or we know the experts.

My immediate first thought, after reading this was: What, exactly, constitutes an expert in testing? How does one become an expert in testing? Added bonus question: How, exactly, would knowing an expert in testing lend more credibility to what I am saying? I’m not the expert in question, am I?

So, how does this becoming-an-expert-in-testing happen? by not challenging the current authority figures? By not trying to come up with something new and/or improved? By not tackling the problems you see and just serenely accepting them (*baah*, said the sheep and continued munching on grass) as a necessary – and unchangeable – part of the world you’re living in?

Let me answer that for you: It happens by stirring the hornet’s nest, by challenging the consensus. It happens by telling people that they’re doing things in an ineffective way, when they are, and then proposing an alternate way. It happens by not just accepting everything that’s being force-fed to you without questioning. It happens by making mistakes, learning from them and then coming back stronger and better equipped than before. It happens by acknowledging the fact your thoughts and ideas could be shot down and torn to shreds in public, and still having the courage to present them despite the risk.

In brief: One does not become an expert (in testing) by being an idle spectator. You do it by taking a stand and making it your very own, personal, responsibility to change the way things are done. And that sure as hell won’t happen if you don’t throw your ideas out there for people to see and have a taste of. Of course, this is just my personal view – your mileage may vary.

Dramatic, I know, but sometimes being an opinionated tester is like the IT-world equivalent of being John Rambo…

As a sidenote: seems I just discovered a defect in WordPress while writing this post. 😛
Sidenote #2: seems I just discovered another defect in WordPress when trying to add more tags to this post (sorry Rob, WordPress refuses to store your name as a tag with Capital First Letters, I’m not trying to be disrespectful here…)

10 Responses to Brothers in Arms, Part 1

Good post. Of course it also helps if you work in an environment that encourages everyone to question 🙂 If it comes down to personalities or those who are brave enough to risk being ‘shot down’ then who knows how many good ideas are being lost? It’d be nice if we could avoid ‘shooting down’ altogether. There must be a gentler way?!

Thank you for the comment, Phil, I couldn’t agree more with that. Fear of authority, or fear of being criticized is an unfortunate thing as I’m sure we would have tons more respected figures everywhere (not just testing or IT) if people weren’t afraid to express their views.

Talking about a gentler way, it’s one of the main reasons why Dalai Lama is my idol. Everyone should read what he writes!

It’s always nice to know that my blog gets viewed and it’s also super nice to know that someone comes away from it feeling charged to make a change. I’m glad you liked the post and I think this blog post is the perfect companion in the shout to make good testing happen.

I like your views on stirring the hive’s nest. We need more people to stand up and say “there is a better way”. And I’m so very glad you found my post useful and I am indeed “honoured” to be mentioned in your blog.

I really like your style of writing and your content so far is interesting too! I’m glad Markus persuaded your to write.

I think questioning and challenging status quo can be done in many ways. Stirring the hornets nest is one technique to do that, though I see lots of other testers changing things through example, mentoring and teaching.

Personally, I see education as a much more effective technique to change, though stirring the nest does have its place.

For me, the key is to sticking to who you are and effecting change in that way.

As for being an expert? I agree you don’t become an expert by being an idle spectator, but the title expert is something people bestow on you and challenging the status quo is only one aspect.
Lots of people put out ideas, that doesn’t make them an expert, but if you actively change peoples lives, well, perhaps that title is worthy.

Thank you very much for your comments – I was hoping someone would bring out a softer perspective.

As a small clarification, I didn’t mean “stirring the hornet’s nest” in a hostile way (ie. irritating people for the sole purpose of irritating people) but, rather, stirring thoughts and emotions simply by being a non-conformist.

Also, I admit I rather simplified the expert part. Making a lot of noise and shoving your own views down people’s throats by force won’t make you an expert – it makes you a true troublemaker (as opposed to being seen as one for the mere fact of not accepting everything without question) – but you do need to put out your ideas for the test and maybe that will have just the effect of people starting to view you as an expert – if you “actively change peoples lives” as you so aptly put it. 🙂

“It happens by stirring the hornet’s nest, by challenging the consensus. It happens by telling people that they’re doing things in an ineffective way, when they are, and then proposing an alternate way.”

For the record, my view are that: (A) it is truly a shame that most testers have never heard of pairwise testing (nevermind more sophisticated combinatorial testing methods), (B) those testers and testing experts that have heard about these approaches tend to pigeon hole them far too much by erroneously concluding “those test design methods can only be put to use effectively when you are faced with [insert far too limiting descriptive words here] kinds of testing problems, (C) combinatorial test design can help make almost all kinds of testing – regardless of phase, underlying code language, type of testing, degree of scripted vs. exploratory focus, etc., significantly more efficient by eliminating unproductive repetition and maximizing coverage efficiently, and (D) the testing industry would be far, far, better off if testers took the time to learn about these test design approaches that generate structured variation from one test to the next (and ensure efficient coverage of different combinations of testing inputs).

Furthermore, the testing community would be well-served with a more science-based approach to determining what methods work better than others. If testers and test managers took the time to do multiple one day pilot projects to measure the results of “trying to find defects with method A” vs. “trying to find defects with method B.” It is difficult to make informed decisions about what methods are effective in given situations if you don’t gather data. I’ve helped teams run dozens of pilot projects that have compared the efficiency and effectiveness of combinatorial test case generation to manual test case selection. The results I’ve seen are dramatic. Some results are summarized here: https://hexawise.com/Combinatorial-Softwar-Testing-Case-Studies-IEEE-Computer-Kuhn-Kacker-Lei-Hunter.pdf

Having said that, my attitude is not “I’m right and everyone else who disagrees is wrong.” My attitude is “This is the empirical evidence I’m consistently seeing. It strongly suggests that using pairwise and combinatorial test design methods can often double the number of defects testers find per tester hour. If your experience with this approach is different, let’s try to understand together why we’re seeing different things. We’ll probably both learn if we understand why we’re seeing different results.” Expressing the data I’ve seen in this way to people like James Bach (who believes there are definite benefits of using combinatorial testing methods in certain situations – but who is somewhat less enthusiastic than I am about how beneficial it can be in a wide variety of testing situations) has been very educational for me, for example. Among other things, I’ve become more aware of the contexts in which I’m gathering the benefits data I’m seeing, etc.

I don’t see any reason why context driven testing could not be combined with combinatorial testing (making it even more combinatorial, heh 😉 ) but I must admit I don’t know enough about combinatorial testing to make any statements further than that.

I have come across the term (and your site in that context) before, and it’s been on my to-be-acquainted-with list for some time now, just haven’t gotten there yet.

I’m relatively confident combinatorial testing will earn a blog posting from me at some point in the future, once I’ve had the time to get to know it better.

I always try to be proactive rather than reactive.:- Once the horse has bolted it is no good trying to close the gate. Thinking ahead or from my experience just doing some actual logical thinking.

Try to be part of a solution rather than part of the problem:- How many times do we hear – we only did this because it is how it is ALWAYS done – even when it does not work. It is no good stirring the hornets nest without having a solution to try and make things more effective and efficient.

Anyway I love posts like yours that make the community think and do a self respective – keep up the great work.