Thursday, October 28, 2010

Testing User Stories - Problem Development Before Solution Development

User stories can be wrong. Business Analysts and Product Owners can be wrong. They can be the best representatives of user interests but they're often still proxy users. The most expensive mistakes to be made in software product development are made when we get the requirements wrong. A quick way to increases the risk of proceeding to solution development on the basis of incorrect requirements is to not prove that they're right. Or, in other words, to not test them.

The only way to definitively validate that the problem statements for a software product are right is to put the working software in users' hands.

User interviews, focus groups, simulations, and mockups are steps along the way, but their purpose is to reduce the time and cost of getting users in front of an experience that is immersive enough to validate assumptions. Allowing users to experience our proposed solutions to their problems clarifies whether the problem is understood.

Generally, we're taking too long to get stories validated - even in colloquial Agile development. Sending a user story through an entire development process in order to validate it is too much. In order to validate user stories, we need to collapse the solution development timeline so that feedback can be had without the expense of the solution development. We commit to the solution development cost once we've validated the problem.

There are two distinct (but inextricably-intertwined) steps that make up a development process that recognizes the value of reducing the time-to-feedback for testing user stories. The later step is the plain-old Solution Development that we use regardless of whether we recognize a preliminary first step. The earlier step is Problem Development.

Problem Development is software development. There's code. There's deployment. There's testing. There's design and analysis. There's process. There's just much more of it in Solution Development (even Agile development) than in Problem Development. And there are different kinds and variations of these activities in each. Problem Development is software development that works hand-in-hand with business development.

Problem Development and Solution Development have different goals. Problem Development is concerned with feeding Solution Development with input that has less risk (at a time when risk is the riskiest).

Assumptions are liabilities. User stories are assumptions until they're validated. To validate user stories, deliver software experiences that propose to solve the problems that user stories describe, and measure the feedback. There's delivery in Problem Development, but not final delivery.

Final delivery is a concern of Solution Development. Final delivery requires long-term sustainability. Solution Development is concerned with long-term sustainability. When we work toward achieving long-term sustainability before we need it, we end waiting until the longer Solution Development process plays out before getting feedback on user story validity. Which is too long.

Problem Development is concerned with short-term sustainability. The reason that it has to be sustainable at all is that the output of Problem Development is the input to Solution Development, and these two steps are ultimately one process that flows together.

It's far too easy to see Problem Development as "throw-away coding", but it's absolutely not. Problem Development is a way to manage some of the biggest risks in software development - risks that often go un-addressed and are frequently realized as wasted effort and resources, and lower productivity. This kind of risk isn't effectively managed with practices that are satisfied by "throw-away" code.

If throw-away code is the output of Problem Development, then Solution Development has to undergo a (mostly) cold start, benefiting from far less of the momentum created by Problem Development. It might be necessary to make the kind of tradeoff that creates these kinds of batch transfer problems, but making a habit of it is a path to a cumulative productivity deficit.

The kinds of practices that characterize Problem Development might look like software development heresy (or at least "Agile" heresy) to many disciplined solution developers. The realities of Problem Development mean that traditional aspects of Solution Development are short-circuited - after all, Problem Development is looking for a shorter circuit.

The Problem Development practices might even seem irresponsible, but recognize whether you're looking through a Solution Development lens. For just as Problem Development practices can seem irresponsible from a Solution Development perspective, applying Solution Development practices to the point in the timeline best suited to Problem Development can be equally as irresponsible.

Problem Development is about testing or validating user stories' problem statements. Solution Development is about building software based on validated problem statements.