Let’s Encourage Live Thinking

In one of my classes, I start things off by challenging the students to “make a diagram of testing.” Some people make complicated pictures, some make simple ones. Some are literal, some are metaphorical. But, many have trouble focusing on testing itself. Testing can be an elusive concept. I like this exercise because it introduces the need for philosophical and analytical skill in the world of testing. If you really want to be a craftsman in this arena, you must learn to go within yourself, summon images and words, and construct practical and conceptual artifacts from them.

This blog post (supplemented by the memories of many others from you and Michael Bolton) led me on a solid 2 hour thinking spree to internalize what it means to be a tester and how best to be one.

This is what I came up with:

– Computers seek to more quickly accomplish what people already accomplish. (-Jerry Weinberg) – Testers need to see if the computer is truly achieving the accomplishments, in a way that is beneficial to people.

How do we do that?

– We find out what is the work that humans did which is now supposed to be aided/replaced by computer – We find out the mechanisms and variables accounted for in the software which define the behavior of the computer so that it will replicate the human work – All the while, we’re forming a representation (or model) – We determine what is the most important – We execute testing with the representation as a guide, with priority given to the matters of import – We communicate the findings

In each of those stages, we gain information from conversations, thought, reading, and interaction with the software.

The most critical goal is to execute the testing in the most important areas to give show-stopper feedback as soon as possible.

He then refined it:

A simplified version, testing represented in 3 steps:

1. Create a representation – Learn the work – Learn the design – Learn the matters of import 2. Test – Use the representation as a guide – Start with the most important – Document purposefully and succinctly 3. Communicate – What you did and didn’t do – What you found – Why it matters

I could critique this, but any critique I could offer pales next to the applause it deserves. Specifically:

His explanation is evidently a product of systems thinking. He refers to static elements and dynamic elements; processes and artifacts; interactions and independent action. It looks like he has extracted this representation from a simulation in his mind. This gives it a multi-dimensional heft.

It includes not just the what’s of testing but also the why’s

It is humanistic. The role and concerns of people pervade his account.

It is itself an exploratory work. It is presented as an evolving concept rather than an eternal truth that is perfect and preformed.

It is expressed in his own words, with his authorial voice. He is accepting responsibility for his ideas.

If any blog post of mine triggers thinking like this, I am very pleased. That’s the major goal of writing about testing. I want to leave a legacy, not of frozen words, but live thinkers.

P.S.

Like too many testers, Nicholas is not yet sure that his ideas matter. After he left the comments, he emailed me:

I made 3 comments on your article “Why I Am a Tester” and realized they’re in bad taste for the subject matter of the article, they’re more self-serving than anything else. They’re still awaiting moderation. I don’t think they don’t belong there and I don’t want to muddy up your blog.

Would you please delete them?

I asked him instead if I could make them the subject of a new post, and he said yes.

Reader Interactions

Comments

Thanks for sharing Nicholas’s reply with us! I couldn’t help but nod while reading this, but also I want to add that a lot of this is a classical (or what I consider to be classical) systems analysis. I am not sure whether that is part of the training even for developers in Western countries, let alone for testers, but I think it should be. When I got my education in Russia 15-10 years ago, it was under the umbrella of systems analysis and implementation – related to software development, but not really anchored to it, and leaning more into the areas of general engineering, modelling of complex business processes and such.

I was delighted to discover testing in the process, because it worked so nicely with the concepts I already loved (and I got into that specialty because I loved it, but formal training certainly helped with the technical vocabulary and understanding of what I do not yet know).

I realise this blog post is about live thinking, not about any specific disciplines underlying it. I guess I wanted to add that having a good base really helps that kind of live thinking, and to encourage others to look up systems analysis and data modelling if they aren’t proficient in them yet. Even when it comes naturally to you, keeping the tools sharp is a pleasure of its own, and in this case very helpful on a job too. In my experience, testers who are good at analysing and understanding complex systems and the problems these systems are trying to solve have a better working knowledge of the project, than anyone aside from (maybe) a system architect. Pretty cool.

I used to ask my prospective testers what they do first with a new app. I got lots of valid high level responses, requirements, user satisfaction etc. However my first action ( having been brought up on 1980s software) is “sit on the keyboard”. I usually got looks of horror from the more educated savvy testers. But – it’s one of the most frequent real world occurrences, and if it fails that, you can send it back for severe rework immediately…..

Jo, I just wonder what is then the information about the state of the product you provide if you send it back just after this test case? Do you think it is the most important bug and that the developers and manager get the information they need?

First thing I would do just turn it on and go through the screens as you learn the model, performing in fact an exploratory smoke test. Then you know whether you can test further. If it works, you can continue based on your estimation of importance and your previous findings.

[James’ Reply: Sitting on it is dramatic and inexpensive. Whatever it reveals, when it reveals anything, is worth investigating. Still, I agree with you that sympathetic testing has more general value than what is essentially fuzz testing, all other things being equal. Of course, nothing stops you from doing both things, and a candidate who mentions both ideas in an interview would please me.]

This is what i usually do when i QA. And i don’t understand why the companies themselves can’t acknowledge this and do it themselves. But i don’t complain because work is work. Great article – quick and simple read 🙂