Monday, March 16, 2009

Today I read an excellent blog post that I just had to share. Jim Murphy is a long-time agile practitioner in startups. He's often felt that there was something missing. In most agile development systems, there is a notion of the "product backlog" a prioritized list of what software is most valuable to be developed next. The breakthrough idea of agile is that software should be built iteratively, with the pieces that customers value most created first. This is a significant improvement on the traditional waterfall methodology.

But, over the years I’ve realized that the toughest problem - the one that matters most and was consistently the most challenging - was figuring out what the product backlog should be.

The backlog is the answer to the question: “What is the most important work we should do right now?” it presumes that you could confidently make that list, and keep it up to date as things change - or at least articulate what you’re building and for whom. Embedded in that assumption is why startups fail. How do you really make the best backlog for your company?

XP and Scrum don’t have much to say - they punt. Its by far the hardest part of the puzzle of shipping successful products and both recommend that you get a customer in the room and ask them to clarify what they want as you go. Well, that’s fine as far as it goes but when you’re a startup and you don’t have customers yet you need a way to bootstrap and that can feel awfully chaotic and wasteful. What’s worse is that as you grow you’ve probably developed some pretty bad habits as far as setting priorities and strategy: like thinking you’re a genius - just because you got funded - and that genius is what allows you to *know* what the market wants.

I remember having this exact same "aha!" moment, auditing Steve Blank's class when we were first building IMVU. Ever since that time, I have struggled to explain how the feedback loop in customer development should interface with the feedback loop in product development.

If you look at the origins of most agile systems, including Scrum and XP, they come out of experiences in big companies. Consider the classic project that was essential to the creation of extreme programming, the Chrysler Comprehensive Compensation System. This was to be a new piece of software to run payroll for Chrysler. In a project like that there are lots of big questions that need to be answered in order to build a working product. But you don't generally have to ask "what problem are we trying to solve?" That's pretty clear. In the case of C3, that was to run payroll for 87,000 employees, who were presumably receiving payroll before the project began. What causes projects like this to fail in traditional software development is that the solution is unknown. Agile is one way to succeed, because pursuing unknowns iteratively is a good way to mitigate risk. What do you do if the problem itself is unknown?

In a startup, rather than think of ourselves as having a marketing department and an engineering department, I now believe it's better to think of ourselves as focusing our energies on unknown problems and unknown solutions. Approaching each of them iteratively is the right thing to do. But the biggest payoff of all can be found when we combine them into one large company-wide feedback loop.

Last year, I found myself back in Steve Blank's class at Haas, this time trying to teach the students about what it's like running engineering alongside customer development. Working with Steve, I came up with schematic diagrams that I hope illustrate this point. (You can see the full deck in my post on Customer Development Engineering or listen to audio from a more recent lecture)

I thought given Jim's prompting it might be useful to post this excerpt. Notice that the unit of progress changes as we move from waterfall to agile to the lean startup. For more on this latter point and why it's so important, consider taking a look at the posts Achieving a failure and Throwing away working code.

8 comments:

I'm pretty excited you've written this post, Eric. I've had at least three conversations this week about the relationship between agile methodologies and customer development. Now, thankfully, I can just send people here.

I agree totally with this post, but one minor clarification on the history of Agile methods.

It's true that at the beginning they did mostly punt on how to determine what to build, but it's not true that they just suggested you get a user in the room to ask. Both Scrum and XP had a role which you could happily call by the modern title "Product Manager". The notion was that however you got them, you needed a person in the room who could make product decisions as needed.

I think Jim Murphy's confusion comes where many people's did, with the XP term "Customer", which was XP's name for the product management role. I'm pretty sure that this term has its origin in Total Quality Management jargon, where each person was encouraged to think in terms of "customers" (who consumed your work product) and "suppliers" (whose work product you consumed). Chrysler was a big TQM shop at one point, so I'm sure this made sense in context, but XP's "Customer" term has caused a decade of confusion.

That naming issue aside, I think it's probably good that Scrum and XP didn't really say much about how to manage products. If they had come up with more system than they did, it would have been something suited to 90s corporate IT projects, not modern startups. As it was, if you followed the XP practices, which included frequent releases, most people in the product management hot seats quickly realized that the users and (real) customers were the often-unwitting source for a lot of information key to product success.

@William - that's a great clarification, thanks for posting it. Reading it again, I don't like the use of the word punt to describe the agile approach. It's tricky to get the tone of a post like this right.

I do think this it is an important for people who want to build an agile startup to understand that customer development is a strong complement to agile.

In my own practice working with startups, I often come across folks who think they are done managing their startup process if they just follow the XP practices. I used to be one of them. Following XP puts you in a good place to start doing continuous improvement. Unfortunately, many startups still use it as an excuse to maintain the artificial separation between the marketing/business team and the engineering team.

That's why I prefer to think of them as two co-equals: the problem team and the solution team. I hope nobody will misinterpret what I'm saying as a criticism of the agile pioneers who made all this possible. In particular, I would not have found myself on this journey without the writing of folks like Kent Beck.

I might suggest you consider employing the Desired Outcome methodology as outlined in "What Customers Want," by Anthony Ulwick to help shape what priorities get on to the product backlog list. This is a fairly simple approach to creating a weighting system using an Opportunity Algorithm.

The algorithm is Importance + max(Importance - Satisfaction, 0) = Opportunity. I won't do the explanation justice so I suggest you grab the book. Simply put it helps product development focus their efforts on addressing Desired Outcomes (the jobs the customer needs done) that create real market opportunities for the product developer.

We are starting to employ it at my new startup (as well as many of the great Lean Startup ideas), and I think it fits nicely into the Agile/Scrum/Lean/4 Steps approach.

@William - thanks for clarifying the role of product management. And I'm thankful that Scrum/Agile doesn't address everything too. :)

I think in my original article I mentioned how there is a wide (tho narrowing) gap between the world of product managers and the agile world. I have certifications in product management, scrum master and product owner roles and have to say that there remains huge gaping holes in our understanding of how these roles come together cohesively.

In the startups I've been a part of the biggest challenges turned out to be what we should be building. By comparison the "how we should build it" was simpler. I've always had the good fortune to work with committed, smart people people with skillz.