Nordic Testing Days

NB: This blog post is adapted from one I wrote for an internal blog post at work, so it has been sanitised for content.

When learning new things, ideas, skills or exploring new perspectives, the image above reflects how I am feeling when I am trying to assimilate and process all the activities I take part in within the testing community. It’s that beautiful and terrifying moment when you are flying and the sun is ablaze on the horizon. It’s knowing you’ve learned a lot, but there is so much left to learn. Perhaps a move one way or another will lead to failure, but as long as you are quick to learn from those mistakes you can be on a well-lit path again.

Over the last month or so I’ve been attending a few events, including ITAKE Unconference in Bucharest, Romania, Let’s Test in Stockholm, Sweden and Nordic Testing Days in Tallinn, Estonia. Each of these events offered something new and different for myself as a contributor to those events. However, it is the other aspects of what they provide that are important.

ITAKE Unconference

ITAKE Unconference was my first time giving a Keynote talk at a conference. This was a huge honour not just to be asked to give a talk, but also the fact that it wasn’t a testing conference. ITAKE is a developer conference. Every attendee is highly technical, lives and breathes the code they write and the tools they use. It made me realise that whilst I have spent a lot of time learning about security, there is so much else to learn. Especially about how developers learn and work, and how applications are crafted.

I spent hours talking about how to build good environments for testing using tools like Docker and Heroku, or exploring how developers think about testing. A lot of it is about unit testing, some of it is about automation. But there are a lot of developers who understand the value of good testing and want to work with testers to make it happen. There is a lot we can do better to support them in this endeavour. These are things we should be doing at Medidata…testing cannot happen or exist in a vacuum.

Yes, there were those that think the role of testing or testers is now defunct, where a technical person can achieve all things they need to on a project. It was interesting to be able to discuss and challenge some of that thinking, where a tester can be a specialist or advocate for testing on their project; rather than someone who executes tests, gathers test results and creates endless meaningless reports. I’m not saying reporting is bad, just the doing the wrong reporting is bad, and unhelpful. It doesn’t add value, nor does it explain to those who don’t test what the value of the testing is.

ITAKE is a hotbed of software craftspeople. People who want to build and develop great software for their customers and clients. The best talk I went to while I was there was one of the other keynotes. Felienne Hermans, of the Netherlands, gave a talk called:

This reflected on her approaches to teaching coding to children. Children learn predominately through play, exploring their environment, and asking questions. It’s something that adults have largely forgotten how to do, or if we haven’t has become more formalised. We’ve turned play and learning into work instead. We can make our learning far more creative through events such as hackathons. We should review, model and landscape our applications inside the environment we are working in. Children do this far more naturally than (some) adults.

She also talks about introducing children to the scientific method, how we observe behaviour, theorise about why the behaviour occurs and make a hypothesis, and then on to experiment in order to prove/disprove that hypothesis. This can be applied to coding as much as any branch of science or other learning.

Let’s Test Sweden

This was a bittersweet event for me. This was my third visit to Let’s Test in as many years, but sadly it will be the last ever Let’s Test in Sweden. The organisers have decided to call it a day. This edition of Let’s Test had a distinctly technical focus, with each session being a three-hour workshop, held over two days.

I ran a workshop called Web Application Security, a Hands On Testing Challenge. We have Security Awareness Training in-house, which covers many of the techniques and tools of security testing, so this follows a similar path. Given the time and space in the office, we should be able to make this learning much more hands on. I try and provide a safe space for the attendees to find problems and ask questions about the application under test so that they can also talk with confidence about security in their own environments.

I attended a number of other workshops, including Alan Richardson’s Evil Tester’s Testing Games of Evil Testing. In this workshop, Alan introduced us to how we can use simple tools such as the browser developer tools to interact with simple JavaScript, API clients and HTTP requests to give us a competitive advantage in debugging web based games.

Whilst the topic might sound trivial, the application and usage of these tools are crucial for testing and debugging modern web applications. Browser based development tools will allow us to do so many useful things:

Viewing the source, both HTML and JavaScript

Debugging JavaScript, via the Console

Tampering HTTP requests

Testing REST services from the browser

Custom headers

Throttling application performance

Emulating other browsers, devices and screen resolutions

During the workshop, we played the games to understand them and their gameplay. We found bugs and fixed them, we wrote code and created cheats to make it easier to win, get massively high scores or infinite lives.

One of the other workshops I attended was Aare Nurm’s Pedal to the Metal. Aare hails from Estonia. His workshop was a fantastic, practical exploration of how software and hardware combine to make products, solving problems and exploring how stakeholder demands can cause issues and constraints with your testing, and how your testing value is perceived and acted upon.

We were given the exercise to test a keyfob for a new kind of car that was coming on the market. We didn’t have access to the vehicle, but only to a prototype to the keyfob. We needed to utilise all our testing savvy to come up with test ideas, find problems, analyse logs and even fix the issues ourselves.

Essentially the keyfob was a Raspberry Pi Zero, with a pin board and wires which could be configured to give different settings. We also had a set of LED’s which would flash according to the function the keyfob was supposed to be executing, including:

Unlock the car

Lock the car

Remote closing the windows

Activating the headlights

Remote boot/trunk opening

Start the engine

We initially observed the behaviour of the keyfob in order to determine it’s function. We were given minimal, but rapidly changing requirements not only for the product but also the business. Essentially if the car failed in the market, the company would go out of business. Hardware and software are so entwined, that even if the product is solid, well made and easy to use if the underlying software architecture is poorly implemented, this can result in a poor customer experience.

Here with my workshop partner Phil, you can observe our testing and learning. This was honestly one of the best learning experiences of my life. Check out the videos here:

Nordic Testing Days

I’ve run testing events before, such as South West Test in Bristol, and SWEWT (South West Exploratory Workshop in Testing). Never before have I helped to run a conference. Nordic Testing Days 2017 was my first adventure in being a conference organiser. It’s hard work, let me tell you.

I’ve been lucky enough to speak at every Nordic Testing Days since 2014, on both using emotional heuristics in our testing, and security testing. Last year I decided not to submit again, to allow new voices to be heard. However, the organisers asked me if I wanted to be a part of the team for 2017. This responsibility came with all sorts of challenges, including organising the venue, social events, curating and selecting the content from the call for papers, interviewing the prospective speakers, organising and facilitating tutorials and talks, as well as solving logistical problems and finding replacement speakers for those that couldn’t make it.

I had the pleasure of facilitating Fiona Charle’s tutorial The Art & Science of Test Heuristics. In this session, we were tasked with coming up with test ideas for two different scenarios, interspersed with both group activities and discussion in the round.

The first activity was to generate test ideas for a number of different puzzle games, such as SmartGames IQ Steps and IQ Fit. The task was to not only solve the puzzles but also identify and utilise heuristics for solving the problems that the puzzles posed. Many questions needed to be asked, including how many ways could we solve the puzzle, what problems or issues did we identify when solving the problem, what oracles can we use when solving the problems? It was no easy task, and one of the teams gave a massive cheer when they eventually solved their puzzle.

The second activity was to generate test ideas from this video:

How many test ideas can you come up with to test the Oh Canada Beer Fridge? The main takeaway from this workshop for me was looking at heuristics as a tool to generate great testing ideas. It’s a complex problem, with no one size fits all solution. Test ideas are our life blood, by being the fuel for our learning and our ability to do our work.

It’s been a week since I have returned from Tallinn and the Nordic Testing Days conference, which has again been brilliantly organised and executed by Grete Napits and her wonderful team in Estonia. Helena Jeret-Mäe led the curation of this years content alongside her colleagues, and without doubt the organising team had certainly raised the bar again.

Santosh Tuppad, Rob Sabourin and Helena Jeret-Mäe out in the Old town of Tallinn

There were speakers from almost all corners of the testing globe, from Canada, UK, Australia, New Zealand, India and of course Estonia. A fantastic achievement! As a result of this breadth and depth of testing talent, it was hard to choose whose talks and workshops to go to, but choose I did.

Warming up

The hospitality and warmth of the conference, and Tallinn is evident. It began with an impromptu walk round the old town with Helena and a whole bunch of other great testers. The amazing architecture, the views from the city walls and the discussion made for a fantastic evening.

Relaxing with the Friendly Tester – Richard Bradshaw

Snap happy! Rob Lambert

But first on to the tutorial days! Bill Matthews and I had already run “Exploring App (In)security” at Let’s Test the previous week, and without a doubt we had learned from that experience. So, we aimed this time to rebalance the session to make it much more interactive and practical at the outset. Many of the challenges of security testing come not only from understanding the threats to applications and therefore businesses, but also understanding how those threats can be translated into real world vulnerabilities, which attackers can then exploit.

Bill Matthews – telling it like it is

Bringing forward those experiences in early, so the attendees were doing practical exercises from the beginning was our primary goal for the day, so that they got the most out of Bill and I, the material we produced and the learning they could elicit from the discussion.

Again, we started out with an exploration of the application under test, but then we burst straight into a group threat modelling exercise!

Threat modelling with these budding new security testers!

After that, all the testers broke into small groups and pairs as we all found ways to exploit the threats we modelled, by exploring the vulnerabilities that might lurk under the covers.

Pairing up with Katrina Clokie (and another tester whose name I can’t remember, sorry)

It was a long and exhausting day. Bill and I were rarely off our feet. We had a great time working with all these fantastic testers. One or two have even got in touch since to ask follow up questions and look for further study. Very encouraging and inspiring! It’s also inspiring me to do a whole lot more in 2016!

On to the first day of the conference proper, and following an interesting keynote by Mart Noorma on the Estonian contributions on space exploration and technology, I had my first major decision to make.

Spinning up your own influence

Katrina Clokie’s workshop “Become someone who makes things happen” was one of the highlights of the conference for me. In this workshop we were challenged to make sense of what our problems are in terms of making an impact on our teams, and influencing the decision making process.

Communicating our beliefs, needs and thinking is a huge problem for testers. I often have issues on influence myself, as I have explained in this blog post: The MEWTation of Communication. So, Katrina’s workshop really resonated with me because of that. We usually worked in pairs or small groups, working through scenarios where our sales skills specifically would be challenged – selling our own ideas, thoughts, and needs in testing.

Katrina Clokie – Become someone who makes things happen

Katrina referred to SPIN (Situation, Problem, Implication, Need) is a sales methodology that focusses on the needs of a customer, and attunes their offerings based on a mutually agreed solution. The problem here is not necessarily getting another person to recognise that the situation you have identified is a problem, that needs resolving, but also the impact to the person you are working with.

For testers there are always scenarios where this technique, and others like it, would be useful. Communicating your thoughts and feelings on acceptance criteria, resolving issues surrounding test planning and estimation, ensuring that you have effective resources and tools to do your job, bug advocacy – the list is endless.

Katrina encouraged us not only to explore the feedback we received, but also attitudes and feelings. Whether you are respectful and caring to those you work with, the difference between aggression and assertiveness, asking the right questions at the right time and using non verbal queues can all have an impact on your influence and ability to get things done.

This was a fantastic workshop that drew the best out of everyone in the room, both new to testing, experienced and hands on, and managers too!

Testing? Thats insanity!

Next up for me was Santosh Tuppad. His energy and enthusiasm for his craft was tested to the full, as Santosh lead us through a beautiful and colourful journey, as he became inspired to begin his own journey by starting Test Insane, his own exploratory test consultancy.

Santosh Tuppad from Test Insane

The great thing about conferences of any sort is that it can bring people together. Santosh and I have been in touch for many years now, but we have never met until we came to Nordic Testing Days. It’s like we have been friends for years, so a warm hug was in order, for this strong man had traversed continents to come and speak for an hour! However his love of testing and learning permeated the social side of this conference completely.

The impressive thing about this talk though was not just Santosh’s clear love of testing as a craft, but also his contribution to the wider community. Sure, Test Insane is a consultancy, but much of the material, tools, mind maps, and papers his team produce are shared across the board – for free! A valuable resource indeed, and Santosh and his team are a very valuable addition to the testing community.

Neil came across with a confidence that belied both his nerves and trepidation at speaking for the first time. He not only talked about the drivers to establishing an exciting and dynamic learning community for testers online, which was the main thrust of his talk – but also some of the psychological thinking that was involved in that process.

Neil talked about the imposter syndrome – where people who are extremely skilled and competent in their chosen field, and yet still feel as if they are frauds, not deserving of acclaim, attention or feel their achievements are of any worth. I think that this is a phenomenon that a lot of people encounter – only the most arrogant of people wouldn’t question themselves occasionally.

However, I feel that this is something that Neil should have no issue with. He is a highly skilled and intuitive tester, with a great breadth and depth of knowledge, and he expresses it well.

Neil’s story is a shared story however, and with Amy Phillips, and their journey to bringing back to life the Europe chapter of Weekend Testing. My involvement in that is a footnote in this story, but I hope to be very much of its future.

Gaming the system

Next up was Kristoffer Nordström and his talk “Gamification – How to Engage Your End Users”. Kris is another tester that I have known about for a while, but had neither met not seen speak. Another great opportunity to learn from someone new.

Kris’ talk was a fascinating exploration of using gamification to encourage the teams he worked with to not only produce great work, but enjoy doing it. By using elements of game thinking and mechanics, developers and testers on his teams were able to contribute to the product by finding bugs and fixing code; and were encouraged to do so by earning points (and points mean prizes).

Kris’s Moomins on tour in Tallinn

Here lies the problem that Kris elaborated on. How do you get people to want to work on code and bugs, but to not want to do just because they are going to earn prizes for their efforts. It’s a complex balance to strike. Renumerating them enough, with branded, high quality pencils, mugs and t-shirts; AND trying to make it fun and exciting for the dev teams.

Kris even gamed the talk, with attendees playing bingo, trying to pick out the key words from the talk. I think I would need to spend some more time with Kris to get more of a handle on this topic, as it is an interesting one. I’ve had to use similar techniques during my time as a trainee teacher and Scout leader to help children become more engaged with activities – collecting stars or badges for examples. Great stuff!

Valuable lessons

Like the rest of the conference, this keynote was another one full of firsts. Rob Sabourin is another tester who I have never met, but had heard many interesting things about. His talk ‘Value Sync’ was an exciting and dynamic discussion about what we value as testers, what people on projects and teams value, and what our stakeholders and customers value – and seeing the relationships between those values.

Rob Sabourin – Value Sync

Rob’s main point was whether the conflicts in these values could be resolved, where one person values low cost over quality, or speed to market over market saturation. There are a lot of elements to balance in teams, organisations and businesses; and testers have a part to play here in expressing what they value, and ensuring that the needs of stakeholders are also met by their testing.

It was a great ending to a long hard day of learning, networking and testing! But it wasn’t over yet.

Lightning in a bottle

So, there was a big sheet of paper on a pillar in the conference lobby – Lightning talks 9pm! I was tired, but there was so much energy in the room. Bill Matthews had already pressed ganged me into speaking – 5 mins of talk + Q&A. So, I contributed the short talk I did at MEWT and compressed it down…trying to pull out the salient points – about how personal identity and problems surrounding being a ‘geek’ in the workplace affects communication and influence.

In the past I have been a 9 to 5 tester, getting to work, doing my work competently, going home. That’s ok! There is nothing wrong with that. I had other interests and needs – I was in a new relationship and/or recently married, or I was playing in my gaming clan. Later, in 2007 I found other interests and got in to Scouting in a big way, which takes up a huge amount of my time. It didn’t stop me wanting to be a better tester, I just didn’t go to many meetups or do much reading, and certainly no conferences. As I have encountered various family crisis recently, I have scaled back my Scouting to focus on those, but my engagement with the testing community has filled quite a lot of that void; and it has been very rewarding.

A lot of this has to do with a number of factors – and one of the major ones for me was working in an environment that allowed me to be the kind of tester I wanted to be. Some of the companies that I have worked with have been less than supportive about attending conferences, worrying about the cost and the value to the business (perfectly valid considerations, I might add). Sometimes, if they allowed it, they specified the kinds of meetings to go to, rather than the testers choice. I don’t think I went to my first meetup until around 2008/9, almost a decade into my career.

I raised a question – ‘isn’t this about bad testers?’ not whether you spend every waking hour testing? It’s a different question, and not one we focussed on. Bad testing is not the same as being someone who doesn’t want to do testing or talk about testing in their spare time.

Neil also talked about introverted behaviours and how they might be a blocker to getting people engaged. It’s a complex problem, and not one easily solved. Except that creating a safe space for learning, either physical or not, that allows anyone to learn at their own speed and their own time can only be a good thing.

Rob Remaining Relevant

Friday morning brought new experiences and new challenges, namely watching Rob Lambert’s opening keynote on the second day of the conference. “Why remaining relevant is so important” reflects on the fast pace of change needed in businesses and services, and our place within that change. Do we sit on our hands and do nothing to meet that challenge and let opportunities pass us by, or do we skill up and start adding lots of value to our teams and businesses.

Ten Behaviours – with Rob Lambert

Whilst it may seem to be basic to talk about how you can remain employable, it talks a lot more to ensure you remain valuable to your team, and continue to be valuable throughout. It’s a challenge we face every day, not only as testers, but as members of a wider development organisation. And ALL of what Rob talks about in this talk is valuable to everyone, not just testers. One of the main points here I took home was adding skills. Add as many skills as you can, become good at them – it might be coding, or using a particular tool, or being knowledgeable about a particular testing approach, or domain knowledge in your organisation. These ALL add value!.

I’d like to reflect further on these points in due course, but it would take too long here to discuss. However I would say this. Rob’s book “Remaining relevant and employable” is a great read. I read it in one sitting by the pool in the Canary Islands, and was one of the main reasons I decided to take a permanent role at New Voice Media. I’ve told this to Rob myself, and I don’t mind telling you now.

Preaching to the unconverted

Lastly, before I wrap up, I wanted to say a word about Katrina Clokie’s talk, which was a last minute substitution to the programme. “Sharing Testing with Non-testers in Agile teams” was a fantastic case study on how Katrina went into a business with little or no testing, little or no agility and was expected to give them all that in a 90 minute training session.

Super sub – Katrina Clokie

Katrina’s experience here was an expression of a depth of knowledge and skill, but also patience, timing, communication, tact and learning. Something we all should pay heed to.

Fantastic Revelations, Amazing Revelations

I’m not going to write much about my own talk “The Testing of Fear” here. I can’t really reflect on this easily in public. Giving this talk was emotionally difficult for me. I had practiced the talk before, at meetups in the UK. All the talks went well.

Due to the unfortunate and untimely death of an early mentor in my career, Adrian Smith, I changed the initial few slides late the previous night. Adrian was important to me, not least because he helped me get my first step on the ladder. His encouragement, leadership, skill and fortitude was an example to all who met and worked with him.

I went to his funeral this afternoon, and I learned a lot about him. As a lad he learned to be a butcher in his home town. He was a Royal Marine Commando, and served his country on many occasions. After leaving the Marines, he served as a police officer, where he met his late wife Deena. After that, he began a largely self taught career in IT – project and people manager, developer, tester, architect, DBA – almost everything you could think of, he could do! We all respected him and loved him. His funeral service this afternoon reflected that, as many of his friends and colleagues joined his family to celebrate his life today.

He was an agent of change – no fear of that.

Farewell…but not goodbye

Nordic Testing Days 2015 was an intense three days of learning and development for me. I hope to be privileged enough to attend again next year, and for years to come. It is a dynamic and exciting conference, with a wide breadth and depth of excellent testers and experiences. I know it will go on being that way! I can’t wait for 2016!

I’m running a tutorial at ATD 2019

Follow me on Twitter

Dan Billing

I'm a software test engineer of 17 years, and recently I decided to go it alone as a consulting tester, founding my company The Test Doctor Limited.
I love testing and all its wondrous variety. I like to help others become better testers by attending events, speaking, blogging and giving training.
Most of my current work focuses on testing strategy across the whole of the clinical trials suite that we build. This includes any kind of testing, from UI, API, performance, security, mobile etc. Whatever needs to happen.
I'm also building on the training, coaching and learning I've picked up elsewhere, and bringing that into my new team.
I enjoy running workshops and speaking, especially in the technical testing and security space; and to a lesser extent the psychology of what testers do.
Hopefully, It'll make me a better tester too!