Software Quality Leader

Post navigation

One question that I always ask about bugs is the impact on our users. What would be the frustration level for those bugs? It’s one thing to find a bug and think what it means to our customers, it’s another thing to be that customer and experience the frustration.

This morning, I was the frustrated one.

I wrote an article for my other site, the Software Leadership Academy. I drafted the article in Word, then opened up the editor in WordPress. It was a different editor than the last time I wrote an article, but on the surface it looked more modern. I experienced several bugs while working with that article, and guess what? The article didn’t save, so I lost about 45 minutes of work. I’ll list the bugs experienced later in this article.

I took another 30 minutes or so to diagnose what happened and how to fix it. I looked all over the WordPress installation for any hints, and disabled several plug-ins that looked related to composing posts. No luck. Searching the web eventually led me to learn that the new editor (called Gutenberg) was from WordPress and is the default editor in the 5.0 release. I have that site set for auto-update, and I received the update in the past couple of days.

To compound the frustration, there wasn’t a way to disable the new editor. I found a plug-in that does allow me to choose between the classic editor and Gutenberg. My frustration is compounded by the fact that I had no warning this was coming, it was a seriously buggy experience, and there was no easy way to revert the change. (I don’t count searching for, installing, and operating a plug-in to be easy).

As promised, here the the bugs that I experienced, in rough order of observation:

Importing from a word document suppressed the space character between words that are separated by newline.

Unclear ability to insert an image “inline” with my text, so text wraps around the image (i.e. left-justified image with text to the right)

Preview didn’t work, it just opened a new tab and gave a “working” spinner for 5+ minutes.

There was no “save” button, but the UI told me every few seconds that it was saving. I believed it, but in the end, the article was not saved.

No clear publish button. I saw a schedule for the article to be published, in 2 days, but no way to publish now. Also, I changed the time to 2 minutes in the future and waited – it did not publish.

Anyway, this article is less about me complaining about WordPress and more about sharing the experience of losing work and being frustrated as a user. We should all consider our bugs from the user’s perspective.

The Software Leadership training course contains a module called Customer-Driven Quality, which helps teams build empathy for their customers and use that empathy to build better software.

Writing automated tests is the easy part. We’ve all seen demonstrations where the sales engineer creates a test on the fly, then re-runs that test. Tools make it easy to see early progress.

Then comes the day-in-day-out usage of automated tests. Tests pass, but the product has bugs in those areas. Tests fail, for no apparent reason, then pass the next time. Your “automation person” runs the tests and understands the results, but if they are gone that day, well no automation. The next product version comes out, your tests are all broken now…

I’ve worked in test automation for many years, on many projects. Each of these projects were in a different stage of maturity for effective automation. Some efforts were no more than having a tool in place, with 1 or 2 people who could create and run the tests. We waited for the tests to be completed so we could add those results in with the manual results.

Other projects had a lot more infrastructure to make the automated tests valuable to the entire development and test organization. Over the years, I’ve added elements of this infrastructure to a set of patterns that I would apply to new automation projects – and those patterns did add value to the new projects. These patterns include standards for version control of the tests, triggers for execution, management of test data & configurations, test execution farms, notifications, results dashboard, and triage helpers.

Along comes MetaAutomation, by Matt Griscom. This book provides a framework which already contains each of those patterns and a few more that were new to me. Matt’s book provides a framework to develop an effective automation program on any software project.

These patterns range across the phases of the development life-cycle for tests. The patterns start with Prioritized Requirements and how these relate to test case creation.

MetaAutomation provides several useful patterns for test cases, including Hierarchical Steps, combined with Atomic Checks allow for reuse of test assets across test cases – and helps isolate failures when they do happen. Several other patterns relate to checking the test results.

The Parallel Run pattern describes how to run many test suites in parallel, perhaps on the cloud, helping to maximize throughput with reduced duration of execution.

Smart Retry and Automated Triage help address the dreaded task of triaging failed tests. Care should be still taken when using these patterns, we don’t want to mask flaky tests by making it easier to deal with false results.

The automated tests generate tons of data. The Queryable Quality pattern shows how the team can create value from these tests.

If you are in test automation, either building individual tests or leading the effort, this book contains lots of lessons learned the hard way.

I’ll wrap up this review with my favorite sentence from this book: “With MetaAutomation, QA helps developers work faster”. The goal of automation is to improve the overall software development process.

Mike Lyles is a leader in QA, and he shares some lessons on his leadership style in this interview. He shares some advice that being a leader is not about you, but about your team. He liked to focus on the success of those that he leads.

Recent events in the US have reminded me how easy it is to blame others for your poor behavior. Real leaders, though, behave in a productive manner, even when they are frustrated. In this article that I wrote for the Software Leadership Academy, I tell the story about when my search for a root cause collided with my confirmation bias.

Post navigation

Software Leadership Academy

Are you interested in enhancing your career into a leadership role? Check out the Software Leadership Academy, an online course designed for you. The Software Leadership Academy provides proven lessons to learn the art and science of management - customized for software professionals (dev, test, ops, etc.)