A simulation to show the importance of backlog prioritization

A successful agile software project depends on a prioritized backlog. You can have a perfectly functioning development team that efficiently finishes one feature after the other, but if these features hold no business value it still is waste.
To show just how important this prioritization is I did a Monte Carlo simulation, using Google Docs spreadsheet. Here are the assumptions I made to create the model:

the business value of a user story is randomly generated from a uniform distribution between 1 (very low value) and 100 (very high value).

the effort to implement this user story is generated by drawing from a standard set of Scrum poker cards with the values 0, 1/2, 1, 2, 3, 5, 8, 13, 20 and 40 (100 and ? are omitted). The drawing was done first selecting an index based on a normal distribution with average 5 and standard deviation 1. This (zero-based) index is used to select a card from the range of poker cards. For example index 5 corresponds with the fifth card which happens to be 5 as well. Index 6 corresponds with the value 8, etc.

the assumption was made that there is no correlation between the business value and the effort needed to implement the corresponding story.

the whole backlog can be delivered in 10 sprints.

during the first 4 sprints the velocity is increasing in a ration 1, 2, 3, 5. This means that the development team is 5 times as fast in the forth sprint compared to the first. After that the velocity stays constant at a value of 5.

With these assumptions I calculated the delivered business value (as a percentage of the business value of the whole backlog) using 4 different prioritization algorithms:

no prioritization. The development team picks up the stories in the (random) order from the top of the backlog.

prioritization using the business value of the stories. Stories with high business value are built first.

prioritization using the needed effort to build a story. Stories with low effort are built first.

prioritization build on the ratio between business value and effort. Stories with a high ratio (high business value, low cost) are built first.

The results can be seen in the following graph:

As you can see from this graph is that not prioritizing your backlog will deliver business value quite late. Since there is no correlation between business value this will be a straight line after the first 3 iterations in which we have a low velocity.

The other extreme is the prioritized backlog using the business value / effort ratio. This is the red line. Here you can see that after 5 iterations you already have delivered 80 % of the business value. For the backlog without prioritization this takes more than 8 iterations!

What looks a bit surprising initially is the difference between the prioritization on business value (the yellow line) and prioritization based on effort (the green line). This is introduced by the fact that in our simulation we have created a bias towards more expensive user stories: the number of user stories with effort lower than 5 is equal to the number of user stories with an effort higher than 5, but remember that the Scrum poker card values are not symmetric: 0, 1/2, 1, 2, 3 versus 8, 13, 20 and 40. That is the reason that in this simulation it is better to prioritize on effort instead of business value.

There is one more interesting aspect to investigate. What is the impact of the standard deviation that we use to draw from our poker cards? I repeated the simulation using the business value / effort ratio for the prioritization and different values for the standard deviation. The result:

The results are easy to explain: when using a higher value for the standard deviation you will get more extremes (either 0 or 40) for the effort to implement a user story. This means that there will be more user stories which require little effort and deliver good business value. These can be build first.

Bottom line: make sure your product owner works hard on a prioritized backlog. Let him quantify the business value of his user stories. As you can see from this simulation half of the effort (and cost) needed to complete the whole project might already be sufficient to fullfil his business needs. This is a huge money saver!

Advertisements

Share this:

Like this:

Related

actually, and correct me if I am wrong, I see a striking resemblance with a practice I observed often to work flawlessly: prioritize whatever has the lowest estimate before all else. First-time product owners would somehow always come up with this, and … it works almost as good as taking business value into account! Why is that?

I think that can only be explained when business value and required effort are not very strongly correlated. In that case you can pick either the lowest estimate with a reasonable chance that the story will have a good business value or the other way around. If the correlation between business value and effort is 100 % then the order in which you pick up user stories doesn’t matter anymore: you always ‘pay’ the same amount of dollars/hours/euros per delivered unit of business value.

[…] Maurits Rijk did a Monte Carlo simulation of a project where implementation is not prioritized (blue) versus prioritized by return on investment (green). Value delivered is the region under the curve. […]

[…] Maurits Rijk did a Monte Carlo simulation of a project where implementation is not prioritized (blue) versus prioritized by return on investment (green). Value delivered is the region under the curve. […]

“the business value of a user story is randomly generated from a uniform distribution between 1 (very low value) and 100 (very high value)”

And already the whole exercise is invalid. In the real world, the value of user stories is never fully understood until they are released and it is anything but uniformly distributed. More like exponential – many are low, some medium, small number are incredibly valuable.