Re: Quality Requires QA

Russ, our handsome, gloriously bearded CTO, pointed out a great article by Andrew Wulf the other day, titled “Quality Requires QA”. In it, Wulf argues against the increasingly popular trend of engineering without QA. Sounds dangerous, like driving a car without a seatbelt.

At Facebook, all engineers write, test, and push their own features to production. Terrifying… – and at the same time really, really, cool! Facebook’s strategy leans on a few key elements: good infrastructure, great culture, and supportive management.

Getting There Takes Time

Strong infrastructure and the right engineering culture take time to develop, which means young companies typically lack both. “Quality Requires QA” makes a great, albeit inferred, point here. If you’re Facebook, you’re serving billions of monthly active users – and that puts unique demands on you.

In other words, the social and political landscape at Facebook is distinct. Its thousands of employees and its being at the forefront of the market make it difficult to draw an apples-to-apples comparison to a still green startup.

All this isn’t to say the rest of us should ignore Zuck and company. There are valuable lessons to be learned. It’s just not as straightforward as monkey see, monkey do.

What We Can Learn

Facebook’s test strategy hints at two values. Clearly, the company prizes speed. There are huge advantages to finding a bug while your editor’s still open. Design decisions are fresh in your mind and misbehaving code is unobscured by further merges and commits. Generally speaking, the faster you find a bug, the easier it is to handle.

The other value I can divine is scalability. Recruiting talented QA engineers is tough as a startup. It’s downright daunting when you’re staffed in the thousands. What’s an easy way to scale the human side of testing? Get engineers to own it.

Engineers should own testing at any company. Ownership instills pride, resulting in quality output. There’s also value in having a proofreader, so most companies still throw a dedicated QA engineer in the mix. At Facebook, they stray. They forego the dedicated QA and double down on automation. Sounds idyllic.

Many Try, Few Succeed

In most shops, test scripts lose their effectiveness at some point. Throwing more automation at the quality monster tends not to work. More specialized tools (like a devoted QA engineer) take up the charge. How does Facebook manage to skirt this? With state of the art infrastructure.

It’s a challenging gambit. Execute successfully, and you win a cool self-serve QA system. The crux of Wulf’s piece is that it’s not all daisies. Even with perfect execution, Facebook is still losing something valuable. That something is easiest to explain by analogy.

I remember typing essays in school that I thought were brilliant. Later, upon receipt of the graded paper, I often found them disfigured by red pen marks. Why? I’d used a nice, quick spellcheck and proofread, also an essential tool. Still, I’d miss things. Only a fresh perspective, a read-through by a person able to step into the shoes of my audience, could catch those other mistakes.

Facebook engineering’s “automate everything” approach yields a pretty nice spellcheck. Its culture promotes proofreading. Yet it’s easy to see how even with all that, as Wulf notes, it could gain a lot from a few dedicated QA people.

Why would a company famed for hiring the best and the brightest forego QA? A conflict of values.

Facebook operates at Facebook scale. It is a Goliath – so large, the only appropriate way to describe its enormity is with its own name. What seem like poor decisions from the outside are likely tradeoffs, endemic to a company its size.

Where We Fit

Manual QA remains exceptionally useful. It can, indeed, also be heavy and slow. What the rest of us should take away from this is “use the right tool for the right job.”

Manual testing is an old tool, and old is neither bad nor good. New tools and strategies appear every day. Making the most of both is key.

Where is Rainforest in all this? We’re one of those shiny new tools. We do a good job of scratching the quality itch in a unique way.

At large or established companies like IBM, Zenefits, and Zapier, we find we fit snugly at the end of the automation tool chain. Where traditional scripted tests start to weaken, we can go strong.

We’re human-powered, but we scale like traditional automation. Hate updating those brittle but critical UI tests? Give ’em to us. For up and comers like Mattermark, inDinero, and Tilt, we can do a lot more.

When you’re bootstrapping, it makes sense to move fast and break things. In those cases, treat Rainforest like a ready-made test suite. Give us a URL, press a button, and we’ll give you peace of mind.

Wrapping up

There’s room to explore in quality management. I’m genuinely excited to see how the tech industry grapples with the issue going forward. Whatever the answer is, I don’t think it’ll be as simple as “follow this one principle”.

While the industry continues its search, I’m stoked to be working with a team that is already developing a new, unique solution. No, really, you should give it a try – it’s free!