Wednesday, January 30, 2013

I hear this question often from people in the pharmaceutical branch, air traffic control, banking and finance - any place where strict regulatory compliance is required, and it came up again at my last CSM course.

Regulated environments are complex, both because of the regulations themselves and the high risks associated with the domain being regulated, which means that Scrum is well suited to developing products in this sector. Let's look at Scrum's building blocks, some objections to using Scrum in a regulated environment, and then, using traceability as an example, let's look at how Scrum can be used to address a regulatory requirement.

Building blocks

Scrum is designed for solving complex problems as a team, without defining any domain-specific process. Scrum gives you several building blocks to ensure that your delivered product meets all of the necessary requirements:

The Product Backlog, a planning instrument

The Definition of Done, an agreement to ensure quality,

The Increment, an additional slice of customer value, integrated into the previously developed slices, and

The Product Owner, a role that takes responsibility for the whole product

Scrum does not tell you to write any particular documents, but if creating and maintaining certain documents is required for your project, Scrum gives you the mechanisms to ensure that they get written and maintained properly.

Product Backlog

The Product Backlog is the source of all requirements, both functional and non-functional, for a project. All requirements belong on the backlog. Non-functional requirements should appear high on the backlog, so that the Implementation Team and Product Owner can discuss the implications early in the project. Usually non-functional requirements will get translated into functional requirements and/or items in the definition of done to ensure compliance.

The Product Backlog is ordered. The top most item will be done first. Then the next one, then the next one, etc, until "all" the backlog items have been implemented. (More formally, the Product Owner decides when "all" features have been implemented).

Definition of Done

The definition of Done is simply common understanding among everyone in the project on what it means when you say, 'This feature is done.' So for instance, if a release note must be written for every feature, then writing the release note belongs in the definition of done. A definition of done typically requires proof that the new functionality has been implemented correctly and that all previously implemented functionality is still working correctly.

The goal is that there should be no undone work at the end of sprint. If that goal is achieved -- and teams that achieve it often do so with the help of automated testing -- then the Product Owner can call for a release at least at the end of any sprint (and possibly more often, depending on the teams engineering practices).

The Increment

Under Scrum, teams deliver solutions incrementally, once slice at a time. These slices are integrated together to create the entire product. The increment at the end of the sprint should be 'potentially deliverable' - there is no undone work, so if the product owner believes there is more value in releasing than not releasing, she can call for a release.

These "slices" are defined by entries in the Product Backlog (and are often called stories). The stories get mapped to acceptance criteria, which can be logged or documented as appropriate. When the acceptance test is green -- and the remaining aspects of the definition of Done are satisfied -- the story is considered done.

Even if it is not practical to actually deliver at the end of each sprint, it is still desirable to have granular control of what is done. It is by implementing features in small slices and by insisting that there be no undone work at the end of sprint that Scrum teams achieve the ability to deliver predictably. By measuring progress in terms of deliverable functionality, they gain the ability to predict when the total package will be finished.

The Product Owner

When is the whole product really done? When can we really release? That is the decision of the Product Owner. The P-O decides which items will be done first, whether a feature is really done, and when the product as a whole can be released (even if release means "released to the approval process.").

Meeting Regulatory Requirements

So how will your project meet the regulatory requirements? Here's how I would do it:

Identify the requirements of the relevant regulations, both functional and non-functional, requirements, non-functional requirements, or deliverables - whatever!

Prioritize these at the top of the product backlog. So in the first sprint or two, the team will figure out how to satisfy these requirements. Probably they will produce new backlog items and/or entries in the definition of Done.

Treat the regulators as stakeholders. Identify their needs. They have (or should have, and maybe need help to develop) a definition of ready - when is a product ready for approval. Try to understand that and you can make it easier to get your product approved. Although it may seem impossible, I would try to reach out to them to start a dialogue, explain how we are developing, what Done means in our process. Regulators are people too.

Does Scrum fit a regulated environment?

One thing that troubles some people is the idea that the project would be released to the authorities every sprint. Releases have high cost, require many signatures, and it just doesn't make sense to release.

OK. Then don't. If there is no point in releasing, then there is no point. However the advantages of the having the product in a releasable state every few weeks are still tremendous. By creating a steady stream of finished, deliverable value, you increase the predictability of your project and dramatically reduce the chance of a major setback near the end.

Why prioritize? I have to deliver everything!

Yes you do! And you have to deliver to a deadline. Now if time is getting short, which would you prefer?

Deliver 'everything' but in questionable quality.

Deliver many features, but the undelivered features are of unknown importance, some get used every day, some not at all, or

Deliver all the features that are used every day or every week and leave out those that only get used seldom or not at all (which makes up 2/3rds of a typical project).

Even if you really have to deliver everything, option 3 is your best choice, for your customer and your reputation!

Handling a regulatory requirement

So let's say you have a requirement for traceability. For every implemented feature, you have to show: where the requirement came from, who implemented it, and show that it was implemented according to the requirements.

A backlog item is a requirement. It has a title and can be enriched as you see fit. Almost every project will enrich the backlog entries with acceptance criteria. And we can enrich them with the necessary information for traceability.

A backlog item serves as a reminder that you will have a conversation (or several conversations) to define what that requirement means. As you get close to implementation, you will have those conversations with the appropriate stakeholders and enrich the backlog item with those decisions.

While you could use an Agile management tool to record this information - and some of them have nice integration with testing tools, let's assume we're using a wiki the product backlog. So the backlog items get recorded on the wiki page.

As the team and the stakeholder(s) agree on the meaning of the story, e.g. the acceptance tests, the screen design, etc, they will record these these decisions on the wiki-page for that story. Design documents and other files can be attached as well. If subsequent decisions change or clarify the requirements, these get logged here.

If there is other information we need to record, like who was involved, this can be recorded in the wiki. Templates help us remember what information to record and a checkbox in the definition of Done -- rigorously applied! -- ensures that we remember to keep the data up to date.

By the time the story is implemented, test cases will be have been created and implemented. Some of those test cases are acceptance tests. Ideally, you would automate them. In that case, it is easy to link the story to the results of the test suite. Even if not, you can still link to the test description and upload the test reports. A product page links to every story that is part of the product, which in turn link to the requirements, which finally link to the test suites and the test reports. Voila! The whole chain is present: Traceability:

And, last but not least, by following your definition of Done, you ensure that you have checked that any manual links are set and that the templates are completely filled out. So every time you implement a feature, when it is Done, you certain that you are release-compatible. When you have implemented the entire backlog, you are certain that the entire result is ready to start the road to approval.

Thursday, January 10, 2013

Join Dan Pink, Steve Denning, Joe Justice, Franz Röösli, Jurgen Appelo and many other speakers as we respond to the Stoos Communique.

It all started one year ago:

Reflecting on leadership in organizations today, we find ourselves in a bit of a mess. We see reliance on linear, mechanistic thinking, companies focusing more on stock price than delighting customers, and knowledge workers whose voices are ignored by the bosses who direct them. All these factors are reflected in the current economic crisis, increased inequity, bankruptcies and widespread disillusionment.

There has to be a better way.

We believe that we uncovered some of the common characteristics of that better way. For example, that organizations can become learning networks of individuals creating value and that the role of leaders should include the stewardship of the living rather than the management of the machine.

This message resonated! The Stoos Gathering was followed an active LinkedIn Group, by the StoosXchange, the Stoos Stampedes in Amsterdam and Phoenix, and numerous Stoos Satellites around the world. The desire to find a better a way is shared by people around the world!

The Communique left one big question unanswered, 'What is that better way?' But we should really say, 'What are those better ways?' Just as there is more than one way to do Agile Software Development, there is surely no single right way to do Stoos.

Stoos Connect! looks for those better ways.

In celebration of World Stoos Day, the Stoos Satellite Netherlands is organising the Stoos Connect in Amsterdam. Dan Pink, Steve Denning, Jurgen Appelo and many others will present their approaches.

In contrast with the previous events, the entire world can participate in this fantastic event! We are setting up live multimedia streaming so anyone, anywhere, can watch the sessions. This is exactly why this event is called Stoos Connect!

The Stoos Network Zürich is organizing two live feeds in Zurich. Liip AG and I am proud to co-host this event in Zurich. Via live video, you can follow all the talks and discuss better ways with other people here in Zurich.