Self-review

What it means

The last section I’ll talk about is self-review. This entails thinking hard about where you are now, the skills and knowledge you have, the tools you use, the techniques you use, everything that is part of you doing your job. And then, take this picture, and measure it against where you want to be, what things you want to be doing. What can you start doing or stop doing? Is the way you do things today, the best way to do it or the most interesting way to do it? If not, then how can you change it? Even with decades of experience, it’s unlikely you know the best way to do everything and have completely mastered every aspect of your role. So there’s always room for growth.

Like a sports team watches video replays of the game to see where they have made mistakes or missed opportunities, we need to retrospectively look at ourselves and our processes and see where we can improve. Or something you just want to do differently to mix things up a bit. Only you will know the parts of your job that are boring to you. Think through, ask people or read about what you can do to change it up?

Is there a manual process you can automate? Is there a new technology or tool you can try out? Can you have a go at part of the process that someone else normally handles? Identify an area you want to change and then look for ways to make it happen, which will probably come back to your reputation and relationships as we discussed earlier.

3 examples

Let me share some experiences I’ve had with self-review.

Early on in my career I was noticing that I was finding great enjoyment in writing
automated code to test our UI, particularly for the complex parts of the app where I really had to solve some tricky problems. So my change I wanted to make was actually just to spend more time doing this kind of testing. So I endeavoured to find more ways to get benefits out of writing automated solutions for test cases, because I enjoyed doing it.

A review might just be to help identify the parts of your role that you enjoy doing and thinking through how you can bring more of that type of work into your role.

Reviewing the way I was being a bottleneck for testing within our team as I mentioned in part 1 of this series led to changes in our team’s processes and this also enabled me to spend some more time learning some developer skills as I tried my hand doing some developer work within the team, making use of the reduced testing requirements. And I liked it.

Perhaps you can find other roles that you can try your hand at to see if you like them and incorporate them into your role. This is the idea behind being a T-Shaped team member, where you bring multiple skills to the team, and so is of course a desirable characteristic for your company as well.

Perhaps the biggest action to come out of reviewing myself in the past year was in my decision to enter the public testing world through blog writing and seeking to speak at a conference. Previously I wasn’t too interested in writing anything or sharing with the general public, I didn’t think anyone would care about what I had to say and I was worried about how people would react, but I saw a great area for growth which would challenge me in a big way to work on my communication skills and respond to any feedback I would get. I’ve now been writing for over a year and have now spoken at my first conference, showing how far I have come in this journey. A huge change from high school Pete who would struggle to do a 3-minute speech with palm cards.

Take-home challenge

My final take home challenge is to take some time, and think through the skills you have, the tools and techniques you use and the processes you use. Where are the boring bits, what do you want to do more or less of? Find the areas you want to change, and make it happen!

Bonus Content

I’m enjoying sharing this journey, so I’ll add in a few extra stories of changes I’ve gone through in my current job.

In my 7 years I’ve worked under 4 different QA team leads. I’ve also worked alongside 6 different testers and am currently in a team of 3 testers. So there have been significant changes in my role that have been somewhat out of my control. It’s quite common that others coming and going from the company provide new opportunities to grow and be challenged in different ways. Each manager had different leadership styles and as I grew in my knowledge, new opportunities and responsibilities were opened up to me. I’ve grown with each manager and I’ve learnt new things from each new tester that I’ve been able to work alongside.

Sometimes being in a company for a while means that new roles open up and you find yourself being asked to try out new roles as the needs arise because you carry with you a lot of knowledge about the company already. This happened for me 5 years ago and I had a brief stint in a DevOps role. My responsibility was to create testing environments for each product team to work on. That was certainly not a role I had imagined going into and there was a lot of unknowns about what I would be doing, plus I had to learn quite a lot in this new role. It was an interesting experience and I definitely learnt some new skills, but in the end it wasn’t for me so I went back to testing.

Perhaps there are opportunities to try out different roles in the company you might be interested to take up. It’s certainly easier to do that in a company you already know, with people you already know then it would be to start a new role in a new company where there’s twice as much to learn. It might not even be a full-time swap, maybe you could just take on part of a new role alongside testing?

Finally, a challenge for you, coming back to where I started the series with Change.org. If there was one thing you could change about your role at work or your company, what would it be? And how would you go about getting others to sign on to your petition to see it happen? Create the change you want to see.

Conclusion

As we conclude: at the start of this series I asked if you currently or have ever become bored or stagnant in your job, like you weren’t growing anymore and needing to change jobs. I then challenged you to create the change you want where you are now so you can start to grow again without the hassle of changing jobs.

So now, having read, I hope you feel refreshed and excited about new opportunities for growth. Be inspired to make changes and improvements where you are without the hassle of changing jobs.

Make use of your reputation and relationships to drive opportunities for change.

Start to consider if there is someone you can share your knowledge with who will challenge your assumptions and someone you can learn from.

Reflect on whether the things you’ve always done are still the best way and the areas of your work you want to change.

I really hope you are more equipped to shake off any feelings of boredom or stagnation and can start to grow once again.

Coaching

What it means

Next, I want to talk about coaching. I mentioned in the previous post that the best way to master something is to learn it well enough that you can effectively explain it to someone else.

Have you ever tried explaining something to a toddler?

You: “Don’t kick the ball in the house”, Toddler: “Why?”,Y: “Because you might break something”,T: “but Why?”,Y:“because you might hit a vase and make it fall down and break”,T: “so?”,Y: “so your mum would be upset if the vase broke”,T: “why?”,Y: “because she likes it”,T: “why”,Y: “I don’t know”,T: “why not” …

You get the point, when you try to explain something, people tend to ask why, which can really stretch your understanding and beliefs. You’ve developed a set of assumptions that the way you do things now is the best way to do it, or that it’s too hard to change. You teach this message to someone else, but they don’t agree, they might say, ‘Why don’t you try writing a program to do that for you?’ … Perhaps you’ll be pleasantly surprised and discover a new solution that works better and say: I hadn’t thought of that, or I don’t know how to do that. Or in other words, that sounds like something I need to change, something new for me to learn. Great!

As you spend time coaching someone, you are being tested each day to deeply understand what you are teaching as you explain it to someone else. It’s a great way to find out if you’ve thought of all the different options or to cement your knowledge.

In a sport’s team, it’s the coach’s job to help show people where they need to improve, and guide them through how to improve. A coach within testing works the same way, pointing out areas for growth, and assisting through the growth. Having someone coach you in testing is quite beneficial for you in helping to identify your weak spots and help you develop a plan to get better. Perhaps the areas of your role that you’ve always stayed clear of will become a little less intimidating, or you are challenged to approach a situation differently. Either way, there are changes for you to make, challenges to take on and new growth to be found.

3 examples

Here’s some examples where I’ve tried this. The first one is actually the reason I’m presenting this series. I have a mentor, Katrina Clokie, who I was connected with through the SpeakEasy mentoring program who has been mentoring me in how to write a talk to

Me, speaking at CAST2016

present at a conference. Just over a year ago, we started connecting via fortnightly skype sessions, discussing what I would need to do in order to speak at a conference. Starting from fine tuning ideas, to learning how to communicate these ideas into an abstract and right through to actually putting the talk together and creating slides. I knew something about these skills previously, but definitely learnt a lot from her guidance.

I learnt how to assess what ideas were worth sharing. I learnt about having a target audience in mind when writing so that the content can be more intentional and actionable. And I learnt about how to communicate my ideas. If you are interested in speaking at conferences, I definitely recommend reaching out to SpeakEasy to get yourself a mentor.

From the viewpoint of being the coach, I’ve found my skills and understanding of writing automated testing solutions have really been challenged when I try to explain my test approaches to my workmates and explain how and why I’ve made the choices I did. Solutions I thought were quite clever and effective or just the only way to do things were challenged as i explained and presented them. I’ve been challenged to investigate whole new technologies and code structures to get a better solution which of course means stepping away from my comfort zone and trying new things which means areas for growth.

I’ve learnt a lot about re-using code, readability of code, maintainability, and much more from the feedback of others as I’ve tried to coach them in the way I’ve written my automated solutions.

The other main source of learning I’ve had from others coaching me is actually indirect coaching through attending conferences, going on training courses and reading blogs. These are all great ways to indirectly have other people coach you as they share their knowledge to a group. There are hundreds of testing blogs, several testing conferences and training courses that are worth going to and provide great opportunities to learn. Which of course, you already know because that’s why you are reading this blog! Keep using these opportunities all around you.

I’ve also experienced this side for learning as the coach as I’ve shared some of my learnings on this blog and had hundreds of people from all over the world read my posts to learn from me. It’s amazing how many people from so many countries want to learn about testing. Communicating my ideas well is certainly an area requiring constant improvement for me.

Take-home challenge

So the take home challenge for this section is probably pretty easy to guess. Find someone you can coach in a skill you are familiar with so they can challenge you and push you to really master it. And going the other way, find someone who can coach you to teach you new skills, whether someone in your company or perhaps in a local meetup, or if needed even indirectly via the internet. Having someone to coach you means they can teach you new skills that they have and challenge you to grow in different areas.

I presented this talk at the Association for Software Testing conference, CAST2016, in Vancouver, Canada in August 2016. This version is slightly updated for an online audience.

Change.org is the World’s largest petition website where people submit something they want to change, then gather the support of thousands of other people and as a result, quite often are able to see that change happen. Just this month in August, a petition to create a wheelchair friendly beach on the Gold Coast has been approved, last year a petition led to the bodies of soldiers killed in the Vietnam War be returned home to Australia from overseas and late last year another petition led to the removal of the Heart Foundation Tick on our food products after claims it was falsely labelling unhealthy food as healthy. Last year, in Canada, a petition led to the government accepting in 10,000 Syrian refugees.

These people saw something they wanted to change, and by doing something about it, were able to make the change happen. That’s what I want to talk about in this blog series, creating the change you want, in your work life by sharing my experience in creating changes in my work life.

I’ve been working as a software tester for 7 years at a place called Campaign Monitor. My job involves a mixture of writing automated solutions and testing and plenty of other miscellaneous bits and pieces as I imagine a lot of you would say about your work. I love problem solving which is something I get to do a lot of in my job and I love helping my team create awesome products through testing. You can find me on twitter at @pete_bartlett.

The reason you need to read this blog series

As I mentioned, I’ve been at my company for 7 years and as I look around my company, it seems I am an unusual case in sticking around in the same job for so long, particularly when not in a managerial role. And this seems to be the same at other companies too.

Graph created mid 2016

This graph of my company shows the number of years people in the Engineering department have worked at the company. A large proportion are under 3 years. Most people start to get bored or feel they aren’t learning any more after 1-2 years and feel they have to move on to something else. Perhaps you feel the same way in your current role, or have felt that in previous roles?

It doesn’t have to be that way! Why not create the change you want to see in the position you are in, and start growing again now? You know how painful it can be changing jobs. Searching through all the job adds, updating your resume, preparing for interviews and then once you find somewhere, you spend the first few weeks or months learning about the context of your new position. Several months go by and you probably haven’t really learnt anything new or had any growth, you’re just adapting to being in a new place. You can avoid all this and use the position you’re already in to create change and start to grow again.

I hope that this blog series inspires you how to do that.

The areas I’ll be addressing

In particular, over this blog series, we are going to look at influencing change using your reputation and relationships, challenging yourself through coaching and reflecting on the work you are doing and wish to be doing.

Reputation and relationships

What it means

First, let’s start with reputation and relationships. In your workplace, I want you to think about a complex part of the product you work with, and then think about who you would go to if you had a question about it… Why did you pick that person? I can say pretty confidently you picked that person either because of their reputation or your relationship with them. Perhaps they have been with the company a long time and have shown time after time that they know what they’re talking about, so you trust them to be able to answer your question. Or, perhaps, they are someone you have a good relationship with, and that’s why you trust them to answer your question.

This trust goes beyond just bringing your questions to them, if they had a new idea for a different way to approach the problem you were facing, you would trust them and try it out right? That’s the benefit of having a good reputation and relationships right? When you hang around in a company for a while, you will start to build reputation and relationships, and you will start to become the person others come to and listen to. You can use this to your advantage. Use it to get others on board when you find new areas for growth and new challenges to take on, and it’s more likely that change will be acted upon.

Perhaps you want to learn a new programming language or learn about security testing or performance testing. If you can figure out the change you want to make, then the influence you have built up in the company will mean people are more likely to listen and agree to try it out.

3 examples

Let me tell you some examples where I’ve done this.

Rewind 3 years at my company and as the 1 QA member in my product team, this is how our testing process looked. At a very simple level, the developers in the team would pick up a task, work on it until they thought it was finished and then i would test it. If I gave it the all clear, it went out to production, if not, it would go back to the developers to fix. Probably a scenario most of you know well.

So, I tested every task that our team worked on before it went out live to customers. This created a bottleneck in the testing stage of the product lifecycle, particularly if I was away. There was also a lot of back and forth with tasks as I found bugs and sent them back to the developers, then they sent them back to me to re-test, then i found something else and so on. We were creating great things, but it was inefficient, so I wanted to change this.

The first step was identifying this to the team, that we needed to do better, they trusted my opinion due to my reputation and relationships with those in the team, and they agreed to try my suggestions out and make it better together.

First change, I suggest we start doing test reviews. Before a developer commits their code and finish with it, I’ll come to their computer and get them to show me it working, I’ll ask questions, do some quick, happy-path testing with some simple variations to catch any immediate blockers and then they can commit it. Then if necessary I would do a more thorough test if needed on the committed code, knowing that I’ve already caught anything obvious, so it’s a much higher chance of not getting sent back to the developers. It’s a nicer way to work as well, the developers would feel annoyed if they had let a simple bug slip through that they should’ve picked up and there’s a sense of pride inherent with showing off the work you did, which gives me a nice way to commend the developers work as well.

This change took a little bit of time for the team to adjust to, as you would expect with any meaningful change. But we stuck it out and overall the reception to the change was great, it stopped a lot of the back and forth with bugs and so people were happy.

Second change, I say to the team, let’s add acceptance criteria to the tasks before you start work on them so you know exactly what you are trying to achieve and how to confirm it works yourself, without me needing to be involved. We use acceptance criteria as a simple checklist of things to test against as the developers were working to guide the way they verify their work and help them catch any issues while they are writing the code, rather than after.
Here’s a simple example of what I mean. The task being worked on might be a page for receiving credit card payments. So in the acceptance criteria, I could say, make sure you check for validation on the credit card input fields and make sure you have appropriate confirmation messaging for a successful payment. There’s definitely other areas that could be tested but those two points should inspire the developer to think through enough of the primary test cases to use for verifying their work without having to spell out every scenario for them.

Again, this was received really well, people had experienced the pain of me catching something at the test review they hadn’t thought of, and while this was still much better than me finding it hours or days later, it was still at the point where they thought they were done and ready to move on and they wanted to find these issues earlier. So knowing these things I would be looking for up front really helped.

So, not only is our team becoming more efficient, but I’ve been challenged to critically analyse product demo’s to catch issues as the developers were showing me their work, and improve in communicating clearly, as I helped provide acceptance criteria to the team for the tasks they were working on.

The other problem we were seeing was that I was a bottleneck as the only tester, so the third change was to shift that by allowing everyone in the team to help out with testing. Now, when a task was ready for testing, anyone in the team could pick it up and test it, to make sure it can work as expected and I no longer needed to have a hand in every task we worked on. Great! Of course this requires training in showing the developers how to test, what sort of things I look for, and still working with them up front to create acceptance criteria for the tasks. But it’s definitely a worthwhile investment.

With the whole team being involved in testing, tasks were getting finished quicker and getting into the hands of customers quicker which is a win for everyone. This change would take the most convincing to get the team on board and takes the longest to properly implement. We are doing well in this area now within my team after trying it for a few months, but there’s still improvements to be made. Some developers are strong believers in the “I code, you test” approach, but to the credit of my team, they were happy to give it a try, and call out for help when they needed it.

Through these changes, there was more learning for me, this time in teaching testing itself, and I find, the best way to get better at something or to master it is to know it well enough that you can effectively explain it to someone else. This is what is happening when you teach someone about testing, which I’ll touch on more in the next section.

My response to the scenario I found myself in was I changed my environment, I learnt new skills and I improved my existing skills by using my reputation and relationship within the team to create the change I wanted.

Take-home challenge

My challenge to you to take home with you for this section is to find an aspect of your job that you want to change, then, use your reputation and relationships to convince others that it’s a good change to make and guide them through it if needed, working through any hurdles that come along the way. You’ll soon find yourself with a new challenge, and new ways to grow, without the hassle of finding a new job.

Day 2

Coaching Testers (workshop)

I chose Anne-Marie Charett‘s workshop for day 2 to learn about coaching testers. Even though I’m not directly in a coaching role, I thought there would be lots of great things to learn and a number of things that would apply to teaching non-testers in my team about testing. The day started on discovering why each of us had come and what we were hoping to learn from the day so we could focus our discussions accordingly.

First up we thought about coaching sessions and how they should not transform into a “How are you going” session. Much more effective is to make the sessions task-based (e.g. let’s work through an activity and I’ll offer assistance along the way) or for the student to come with a question.

The next lesson was a great approach to teaching new techniques or skills by starting with applying it to a familiar, non-testing situation. For example, creating a mind-map of your family. This takes away the fear of getting it wrong since it’s known and familiar, and greater focus can be put on the technique or skill being learnt.

In understanding what things to coach and how to go about coaching them, we need to understand the context of both the coach and the student. They will each have 2 images of themselves, one of how they view themselves as a tester, and the other as how they view themselves as a coach/student. As well as many other factors that need to be considered in knowing that different approaches work for different people.

The day was broken up with a number of movie clips to show different coaching styles. First up was the scene in The Matrix where Morpheus asks Neo to show him the Kung-Fu he has just learnt. Throughout the scene, more pressure and direction is added from Morpheus as Neo progresses through the task. It starts of easy, then gets harder and more physical as Neo is pushed further and further to learn the lesson Morpheus was teaching him (the rules can be broken).

Next we watched a clip from The Blind Side where Sandra Bullock’s character was better able to teach her foster son how to defend in Grid Iron better than the team coach. This was because she better understand the needs of the student and what motivates them (in this case, family). The question that he needed answered was “Why” not “How”. Trust was also a big factor in the success of the coaching efforts.

Taking this lesson further, as a coach you want to find out what people’s drive is, why are they in testing, how did they get here and where do they want to be? Then the coach can focus on directing them on how to get there and where they need most help, based on the journey they’ve had so far.

Our next video was from The Magnificent Seven, where a young character was challenged to match his speed at drawing his weapon against the team he hoped to join. This was more about putting people out of their comfort zones and seeing how they will respond to pressure. It was a way to show this man that his skills weren’t up to scratch, and that he would need to lift his game.

Following this we talked about Socratic questioning, which is all about not directly answering people’s questions but instead asking questions in return to help lead them to discover the answer for themselves. This is a very powerful tool in coaching, an answer you come to by yourself is much more powerful and memorable than one you were told. But it can also be taken the wrong way and be demotivating when people feel like they never get straight answers.

The coach will need to help manage the way people feel about themselves and base their coaching on the students needs, energy and context. There isn’t a one-size-fits-all approach.

Our final movie clip was from Million Dollar Baby where Hillary Swank’s character is trying to convince Clint Eastwood’s character to coach her. There was certainly reluctance in this coaching partnership, but biases had to be overcome and the student had to reflect and show they did indeed have the passion and desire to not give up, even though it would be hard. Sometimes we too might not feel comfortable at first about coaching someone or being coached.

2 final points were made around managing expectations and reflecting on progress. Manage people’s expectations to learn something quickly by showing that confusion is just a learning state and is not a weakness or lack of ability. It’s also powerful if you can show someone how much they have grown already to help give motivation to continue on that journey.

The day finished with a practical exercise of simulating a coaching session between a Coach, a tester and a developer, trying to come up with acceptance criteria for a payroll system. This was great to put into practice some of the things we had been learning throughout the day and see how even in controlled environments, people have quite different needs, opinions and expectations and so coaching will need to adapt all the time to suit.

Summary

I had a great time at Australian Testing Days 2016 and am looking forward to going again next year! I’d also love to hear what other people got out of the conference or if you have a different view point to anything discussed in either of my posts about it.

On May 20-21 I went to the inaugural Australian Testing Days Conference in Melbourne. The first day involved a series of talks, mostly on sharing experiences people had in testing and the second day was an all-day workshop on test leadership. This post outlines the key messages from the session I attended and the key things I learnt from each one.

Day 1

Part 1 – What you meant to say (keynote)

First up, Michael Bolton discussed how the language we, as testers, use around testing can be quite unhelpful and cause confusion for those involved. For example, automated testing does not exist. You can certainly automate checking, which is mainly regression tests of existing behaviour. But you cannot automate testing, which is everything someone does to understand more about a feature, giving them knowledge to decide on how risky it is to release it. The way we communicate what we do will impact what others understand it as and then expect of us. Similarly, it is important to understand the language others use when asking us to do something.

Another helpful lesson was that customer desires are more important than customer expectations. If they are happy with your product, it doesn’t matter if it met expectations. If I don’t expect the Apple Watch will be of much use to me, but then I try it and discover that I love it, my expectations weren’t met, but my desires were, and it’s a good result. Similarly, users might expect something totally different to what you produce, but if they discover that what you made is actually better than their expectations, it is likewise a good result.

Lessons learnt:

Be clear in communication of testing activities to avoid ambiguity and misalignment.

Seek the underlying mentality behind people’s testing questions

Part 2 – Transforming an offshore QA team (elective)

Next up Michele Cross shared on the challenges she is facing in transforming an offshore, traditional and highly structure testing team into a more agile, context driven testing team. The primary way to achieve any big change like this is in creating an environment of trust, which comes in 2 ways. A cognitive trust is based on ability, you trust someone because of their skills and attributes. For example, trusting a doctor who has been studying and practising medicine for 20 years that you have just met to diagnose you. An affective trust is based on relationships, you trust someone because of how well you know them and how you have interacted with them in the past. For example, you trust a friend’s movie recommendation because of shared interests and experiences, not their skills as a movie reviewer.

To help establish this trust and initiate change, three C’s were discussed.

Culture – Understanding people and their differences. The context that has brought people to where they are now will greatly shape how they interact with people. Do they desire structure or independence? Are they open to conflict or desire harmony? Knowing this can help inform decisions and approaches.

Communication – Relating to other people can be just as hard as it is important. Large, distributed teams bring with them challenges of language, timezones, video conferencing etc… It is crucial to find ways to address these concerns so that everyone is kept in the loop, aligned on direction and is able to build relationships with each other.

Coaching – Teaching new skills through example and instruction. Create an environment where it is safe to fail so people feel comfortable to grow. Use practical scenarios to teach skills and get involved yourself.

Lessons learnt:

Consider the cultural context of people you are interacting with as it will shape how to be most effective in in those interactions

Learning through doing, and doing alongside someone is a great way of learning

Trust is built by a combination of personal relationships and technical abilities

Part 3 – It takes a village to raise a tester (elective)

Catherine Karena works at WorkVentures which is all about helping under privileged people develop like skills and technological skills to help them enter the tech workforce. She talked about how to figure out what skills to teach by looking at where the most jobs were in the market and the common skills required. This includes both technical and relational skills as they interact with structures and other staff in companies.

On a more general note, a number of characteristics of what makes a great tester were highlighted to focus on teaching these skills as well. A great tester is: curious, a learner, an advocate, a good communicator, tech savvy, a critical thinker, accountable and a high achiever. When it comes to the learning side, a few more tips were shared around teaching through doing as much as possible, making it safe to fail, using industry experts and building up the learning over time.

Some interesting statistics were raised showing that those trained by WorkVentures over 6 months were equal to or greater in performance and value compared to relevant Uni graduates when rated by employers.

Lessons learnt:

Relational skills can be just as, if not more, important than technical skills in hiring new talent.

Learning in small steps with practical examples greatly improves the outcome.

Part 4 – Context Driven Testing: Uncut (elective)

Brian Osman talked about his experience growing in knowledge and abilities as a tester and how greatly that experience was shaped by testing communities. He explained how a community of like minded people can help drive learning as they challenge each other and bring different view points across.

A side note he introduced was a term called ‘Possum Testing’ which is how he described “Testing that you don’t value, motivated by a fear of some kind”, for example, avoiding using a form of testing because you don’t understand it or how to use it. This is an idea that many people would understand, but could perhaps find it hard to articulate and discuss. Giving it a name instantly provides a means to bring it up in conversation and have people already have a good idea of the context and any common ground in thinking.

Lessons learnt:

Naming ideas or common problems is a helpful way to direct future conversations and bring along the original context

When looking to improve in a certain area/skill, find a community of others looking to do the same thing.

Use these communities to present ideas, defend them and challenge other’s ideas. Debates are encouraged

Part 5 – Testing web services and microservices (elective)

Katrina Clokie (who also mentors me in conference speaking) spoke about her experience testing web services and microservices and a previous version of the talk is available online if you are interested. Starting with web services, she pointed out that each service will have different test needs based on who uses it and how they use it. Service virtualisation is a common technique used in service testing to isolate the front-end from the inconsistencies and potentially unstable back-ends. Microservice testing puts another layer in this model.

Some key guidelines for creating microservices automation were presented, claiming that it should be fit for purpose, remove duplication, be easy to merge changes, have continuous execution and be visible across teams.

An interesting learning technique was present called Pathways which can be found on her website which list a whole bunch of resources for learning about a new topic. They are a helpful way of directing your learning time with a specific goal in mind.

Lessons learnt:

Make use of Katrina’s pathways for learning about a new area (for myself or as recommendations to others)

Get involved in code creation as early as possible to help influence a culture of testability

Write any automation with re-usability and visibility in mind

Part 6 – Test Management Revisited (keynote)

Anne-Marie Charett finished up the day sharing some reflections and approaches she implemented from her time as Test lead at Tyro payments. She started by asking the question, do we still need test management? Which has been asked a few times in the community already. The response being that we do need a testing voice in the community to go with all the new roles and technology coming through like microservices, and DevOps. This doesn’t mean we need Test Managers who deal with providing stability, rather Test Leaders who can direct change. She talked about using the “Satir change Model” to describe the process of change and it’s effect on performance.

She brought a mentality to transforming Tyro to have the best test practice in Australia, and was not interested in blindly copying others. There is certainly benefits to learn from the approach others take, but should be assessed to meet your company’s environment. She discussed a number of testing related strategies that you might have to deal with: Continuous delivery, testing in production, microservices, risk-based automation, business engagement, embedding testing, performance testing, operational testing, test environments, training and growth.

The next question was how to motivate people to learn? Hand-holding certainly isn’t ideal, but you also probably can’t expect people to spontaneously learn all the skills you’d like them to have. This needs coaching! And the coaching should be focused around a task that you can then offer feedback on afterwards. Then challenge them to try it again on their own.

An important question to ask in identifying what skills to teach is in highlighting what makes a good tester at your company, because your needs will be different to other places. She then finished with a few guidelines around coaching based around giving people responsibilities, improving the environment they work in and continuing to adapt as different needs and challenges arise.

Lessons learnt:

Any practice/process being used by others should be analysed and adapted to fit your context, not blindly copied.

Be a voice for testing and lead others to make changes in areas they need to improve on

Think about what makes a good tester at my company and how I measure up

Help prepare the organization/team for change and help them cope as they struggle through it

That’s a wrap for Day 1, find my review of Day 2 here, where I took part in a workshop on Coaching Testers with Anne-Marie Charrett

I was recently asked how we measured quality in our product and was immediately cautious about what to say next. I had these warning bells ringing in my head:

“High code coverage doesn’t equal a high quality product if the most important code isn’t covered”

“A low bug count doesn’t mean anything if those bugs are all critical, similarly a high bug count might not be bad if they are all trivial”

But just because there are bad ways to try and measure quality, doesn’t mean there aren’t helpful ways as well!

Watching the trend

The most important thing to factor in when trying to measure quality is the trend over time.
5 bugs raised this month is good if the last 3 months averaged 30 bugs each, however if they averaged 0 bugs each month, it’s a concern worth investigating.

Numbers on their own don’t mean a whole lot, there will always be a need to look beyond the numbers to see the type of bugs you are finding, and what impact they have on the customer. The numbers can influence when and where you should be investigating.

Therefore, the measures you decide on need to be tracked over time to see where you are making improvements worth celebrating and where you are getting worse that needs addressing. Though regardless of trends, this is only part of the picture of the quality of your project. Manual verification will always be needed in parallel.

What should I measure?

In my view, the most important thing you can measure to give you an indication of the quality of your project and where to focus your efforts is:

Total Open Bug Count, grouped by priority

A graph of total open bugs, split into the different priority levels (e.g. Critical, Major, Minor and Trivial) grouped by week or day. This will let you know if you are making a dint in your backlog of bugs, as well as the trend in each category of bugs.

This will help you answer the following questions, which you should then decide on how to respond based on your project’s needs:

Are we creating more bugs than we are closing?

How fast are we resolving our critical and major bugs?

Are we creating more critical/major/minor/trivial bugs than we used to?

Do we need to take some time out to tackle our bug backlog?

Other Measures?

Here are some other measures that could be helpful, but I would do these as a secondary focus to the one above as they are mostly covered.

Total Bugs Raised, grouped weekly by priority.How: Count how many bugs were raised this week.Ask: Are we raising more bugs this week than we normally do? Are we raising more high priority bugs than we normally do?

Average Bug Resolution time, grouped weekly by priority.How: For the bugs resolved this week, measure how long they were open for and get the average.Ask: How long is it taking us to fix critical bugs? Are we getting slower or faster at resolving bugs?

Open Bug Count By Feature, grouped weekly.How: Break the total open bug count down into the feature area affected.Ask: Are certain parts of the project more problematic than others?

Is there a different measure you use to track quality in your project? I’d love to hear about it in the comments below.

Like this:

The first sign of smoke is always an early indicator that fire is not far away. It’s the first warning sign. So too in testing, having Smoke Tests that quickly step through the main pathways of a feature will let you know if there’s a significant problem in your codebase.

What makes a good Smoke Test?

There are 3 factors which make a good smoke test: Speed, Realism and Informative.

Speed

If there is a fire coming, you want to know about it as fast as possible so you have the most time possible to respond. Your tests must be fast enough that running them with every build/iteration will not cause frustration or significantly hold up progress.

If there is a problem, these tests will tell you, quickly, that you need to stop whatever else you might’ve been doing and respond to the issue.

Realism

If smoke is coming from a contained, outdoor BBQ in your Neighbours backyard, it doesn’t matter. You don’t need to do anything about this. But if a tree in your neighbours backyard was on fire, this is much more concerning. The Smoke tests must only look for realistic and common usages of your feature, where you care if something breaks. If the background colour selector on your form creation tool is broken, that’s probably not as crucial to worry about, compared to if the form itself didn’t work.

So, prioritise your tests to only cater for the key user workflows through your feature, and leave everything else for other tests to cover. This will also go hand in hand with designing tests for speed.

Informative

If you see smoke, the first thing you will want to know is how far away it is and how big it is. This will dictate how devastating it will be and what sort of action you need to take. So to the tests you write must be informative about where the problem is, how you can recreate it and how important it is to fix.

Smoke tests are much more useful if they can not only tell you that something is wrong, but also where the problem is, how important a problem it is and how to fix it.

An Example

Since the company I work for (Campaign Monitor) is an email marketing company. So a key scenario or User Story we would want to target in a Smoke Test is:
“As a user, I want to create a campaign, send it to my subscriber list, check the email got through and that any recipient interactions are reported”

So, the steps required for this smoke test are:

Create a campaign, set the email content and the recipients

Send the campaign to some email recipients

Open one of the received emails and click on a link

Check the report shows the email was opened and clicked

This test covers an end-to-end user story that is a vital part of our software, hence if any part of it fails, we need to know immediately and fix it. It is fast and runs in under a minute so doesn’t interrupt workflow. It does not add in any extra steps or checks that aren’t helpful to the target user story. And it is written with logging and helpful error messages so that any failures point as close as possible to the problem area.

Putting it together

Once you’ve created your smoke tests, put them into use! Run them every time you change something in your feature and they will quickly tell you if anything major is wrong. It’s so much better to find a major issue as early as possible and reduce time lost searching for the problem later on or building more and more problems on top.

Smoke tests shouldn’t be your only test approach as they only cover the ‘happy path‘ scenarios and there are plenty of other areas for your feature to have problems. But it is certainly a good starting point!

I’d love to hear how you use (or why you don’t use) Smoke Tests in your company!

Like this:

If you give a customer great service and they enjoy their interaction with your business, you are building up loyalty in that customer. And loyalty is very important! I was reminded of this as I visited the barbershop I have been using for the past 10 years, despite moving further away. I’d like to share my thoughts on customer service and loyalty.

For me at the barber, I experienced great service by a welcoming atmosphere, greeted by a smile. He remembers my name and what’s going on in my life to ask how things are going. He remembers the way I like my hair cut, but doesn’t assume anything. He is happy to bend ‘rules’ of opening times or how long he spends on doing my hair or what happens if I forgot my money and it might be a simple scenario but it all makes for a great experience that keeps me coming back.

Characteristics of a loyal customer

A loyal customer will:

Put up with inconveniences like further/longer travel and limited support to return to your business because that’s where they feel comfortable. I travel past several other hairdressers to get to my preferred barber and put up with a limited window of opening hours I can actually get there outside work because I like it there and I feel comfortable going there. I only visit a couple of times a year, but he remembers my name and what’s going on in my life. He knows how I like my hair cut and is always friendly to deal with.

Not be swayed by sales and gimmicks other companies might use to try and draw their business. Unless it’s a huge difference, it doesn’t matter if they can save money elsewhere, people are resistant to change and so will stick with what they already know and love. I could get a cheaper haircut, but I like the way my barber does it and I value the relationship I have with him.

Be an ambassador for your business as they tell their friends about how great your business is and encourage them to try for themselves. Word will quickly spread when your loyal customer tells their friends about you, who then tell their friends and so on. One positive customer interaction can have great power in promoting your business, just as one negative interaction can cause great damage. I would happily recommend my barber to others.

What makes a loyal customer?

So how do you convert a regular customer into a loyal customer? Sometimes you might be able to buy loyalty through Rewards or Loyalty programs or bulk discounts. But this is short-lived as soon as someone else offers a better program. Instead, it’s all about the service you provide.

Make your service personal, showing that you value the customer and their individual needs. Remember the person when they return to your business so they know their presence was felt and their follow up visit is easier because they don’t have to repeat context like their name and preferences. Do this and you will go a long way towards building a sense of loyalty which will benefit your business in the long run.

What about you?

What companies are you loyal to and what was it about them that influenced you? What do companies need to be doing better at in this area?
I’d love to hear your experiences in the comments below.

Share this:

Like this:

Being the Champion for Quality doesn’t mean being the only one who cares about it, the only one who does anything to improve it or even the one who is best at improving quality (though that might be true). The Champion for Quality is the voice of motivation and direction for the team to create a high quality product. That’s how I picture the role of a QA/Tester.

You’re the Voice

ASK what people consider as important for quality so that you know what areas you should focus on to deliver this. Is it more important for the product to be fast than accurate? Is it expected to handle a large amount of users? What sort of failures are acceptable and what are unacceptable?

TEACH people that quality matters. In order to achieve the level of quality desired in the target ideas identified above, the whole team needs to be on board. Show people the full story, how a lapse of quality in 1 area eventually leads to a negative customer experience and then a negative experience for the company.
Teach them about cutting corners now leading to more work later on to do it right.
This shows them the importance of their work and makes them feel a bigger part of the company and encourages them to put in the extra effort now, not later.

TRAIN people in how to build a quality product (by your definition of quality) by giving them the tools they need, the test cases, the conditions to experiment with, a peer to review and a clear understanding of what they need to do to ensure they are creating a high quality product.