Friday, July 3, 2009

One of the questions that often arise is in regards to Agile metrics. What metrics should we capture for our Agile or Scrum teams? The answer varies based on the goals of the team, project, and organization. With that said, below are some of the most commonly used metrics for Scrum teams.

For teams using an iterative Agile process, like Scrum, the most common metrics revolve around story points. Commonly, Scrum iterations (sprints) are comprised of a set of user stories with each user story given an estimate in story points.

At the iteration level, the team commits to the stories they can complete in the iteration. The sum of the story points for these stories is the projected velocity for the iteration. Once the iteration is completed, the sum of the story points for all of the completed user stories is the team’s actual velocity for the iteration.

Teams may also decompose stories into tasks and these tasks can be assigned estimates as well. These estimates may be hours to complete, due date, or any other applicable unit of measure. This decomposition and task level estimation is optional; I recommend this level of granularity to be based upon the needs of the team and project.

Metrics for the Project

For a project that has a set beginning and end state, the project team can initially spend a bit of time creating all of the high level user stories and providing initial story point estimates. Once this is done, the team has an initial summary total of story points for the project.

Table 1 displays the metrics covered at the project level.

· Baseline Story Point represent the remaining story points at the start of the iteration

· Story Point Updates represent story points that are added or removed to the overall project during the iteration

· Story Points Completed represent the story points completed for the iteration

· Cumulative Story Points Delivered represent total story points delivered to date

· Story Points Remaining represent the remaining story points at the end of the iteration

· Ideal Pace represent the pacing of the project based upon the story points remaining and estimated velocity of the team. In this example, I show it as it was at the start of the project, however, this should be updated often. Updating at the end of each iteration is ideal.

Table 1 - Project Metrics

Iteration

Baseline Story Points

Story Point Updates

Story Points Completed

Cumulative Story Points Delivered

Story Points Remaining

Ideal Pace

0

1200

0

0

0

1200

1200

1

1200

196

60

60

1336

1125

2

1336

210

75

135

1471

1050

3

1471

22

67

202

1426

975

4

1426

-78

75

277

1273

900

5

1273

38

88

365

1223

825

6

1223

30

79

444

1174

750

7

1174

-106

82

526

986

675

8

986

22

90

616

918

600

9

918

-50

100

716

768

525

10

768

-10

92

808

666

450

11

666

25

95

903

596

375

12

596

40

98

1001

538

300

13

538

-30

108

1109

400

225

14

400

-40

119

1228

241

150

15

241

-23

105

1333

113

75

16

113

-15

98

1431

0

0

Figure 1 displays the Project Burn Up and Project Burn Down chart for the same project. The Burn Up line displays the story points completed. The Burn Down line displays story points remaining. For organization that are familiar with EVM type tracking, these Burn Up and Burn Down values can often be translated to EVM measures.

Figure 1 - Project Burn Up and Burn Down Chart

Metrics for the Iteration

The Iteration Burn Up and Iteration Burn Down chart is similar to the Project Burn Up and Project Burn Down charts, but the iteration chart tracks the work at an iteration level. In Figure 2 we are tracking the iteration based on hours completed and hours remaining. We can see that at the end of the iteration, there was 94 hours work of work that was not completed.

Figure 3, Figure 4, and Table 2 display metrics captured from an actual Scrum team. These metrics are also useful for on-going teams or projects that do not have a set end state.

Figure 3 – Team Completion Percentage

Figure 4 - Measure of Velocity over Time

Table 2 - Team Statistics per Iteration

Measuring Business Initiatives and Value

To measure work on business initiatives, we can summarize story point allocation based on initiative (see Figure 4). This allows the business sponsor to identify where the effort was spent on their project.

Table 3 - Second Quarter Work Breakdown

Story Points

Percentage

Foundational

78

22%

Maintenance

9

3%

Optimization

168

48%

New Product

98

28%

Additionally, the business teams can also assign business points, similar to story points. Whereas story points measure the complexity of a story to implement, the business points measure the business value of a story once it is implemented. This display of value and effort can help the business determine implementation priority and goals.

· Developing processes for non-technology based teams and organizations

As a management consultant, Richard has coached and mentored clients on the adoption and implementation of Agile and Scrum. Richard also leads Excella’s Agile Center of Excellence. Currently, Richard is working to bring Agile to the Federal government and is collaborating with the Office of Personnel Management (OPM) on Agile programs.

A graduate of Virginia Tech, Richard has authored several publications on project management, presented at Agile and PMI sponsored industry events, and is a member of Mensa. Richard is a certified Project Management Professional (PMP), a Certified Scrum Master (CSM), and a Certified Scrum Practitioner (CSP). Richard is an active member of the Project Management Institute (PMI), Agile Project Leadership Network (APLN), Scrum Alliance, and Agile Alliance.