Agile Journeyman

Pages

Monday, June 4, 2012

At the May DC Scrum user group, we had a workshop on retrospectives. We covered six different techniques to keep retrospectives fun.
For all the techniques we followed this structure:

Set the stage: This entails ensuring that everyone feels safe to express their opinions. A good way is to start out with the prime directive:"Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand"

Gather data: This is where we create a shared pool of data to ground the retrospective in facts instead of opinions.

Generate insights: This is where we observe patterns, determine root cause, and build a shared awareness of the situation.

Decide what to do: This is where we move from discussion to action and come up with a plan to resolve the top 2 or 3 items.

Close: This is where we review the findings, reiterate the action items, and appreciate everyone’s participation.

Monday, May 21, 2012

In my previous post, I described the changes that Jez Humble believes an organization needs to go through to enable DevOps. In this post, I’ll describe why Jez is encouraging organizations to align around products instead of projects.

On average 29% of an IT budget is spent on new system development while 71% is spent on maintaining existing systems and adding capacity to those systems. This is because organizations do not retire applications. A lot of money is being spent on maintaining mission critical applications. Some of these applications are even running on software no longer supported by the vendor. More money is being spent on applications that have part or all of their source code missing. This is mainly because once a product is delivered to production, the project team that built it is disbanded and team members move onto different projects.
Organizations have no way to rationally decide what to retire. Everything is rolled up into one budget and there is no clear way of deciding what systems are generating value and which are not.

Amazon applies the motto “You build it, you run it”. This means the team that develops the product is responsible for maintaining it in production for its entire life-cycle. All products are services managed by the teams that built them. The team is dedicated to each product throughout its lifecycle and the organization is built around product management instead of project management.

Using this approach, teams can measure cost and value delivered per product team.
Teams can rationally determine what to prioritize in their product backlog and teams can self-manage using a lean startup methodology.
The architecture groups becomes responsible for regulating and monitoring system-level attributes
while the PMO regulates and measures to prevent “market failure” and external factor. Some Ops team members will move to work with the product teams while other Ops team members are no longer responsible for supporting the application but instead manage the infrastructure that applications run on. It is kind of like an in house infrastructure as a service with Service-Level-Agreements for the product teams.

Using a product management approach, the organization now has budgets for product teams instead of new development or maintenance. Budgets are no longer lumped together, but are instead dedicated to specific product lines. Each product team needs to make the case why their product needs funding based on the business value that it generates compared to total life-cycle cost. Using this approach, organization can now make business decisions as to which products to innovate on and which to cut or retire.

Adapted from Enterprise DevOps: Breaking Down the Barriers between Development and IT Operations @ADP East 2011

Culture is the most important part of DevOps. We need to make sure development and Ops are not treating each other as rivals and are instead working together frequently and efficiently. Ops need to be involved early on in the development process. They need to be in the inception meetings and in the demos. They need to know what is coming down the pipeline and provide their requirements in terms of monitoring, scalability, archiving, etc… Developers also need to rotate through the operations department. They need to be on call in case of production problems. They need to feel the pain and impact of not delivering production ready applications. Incentives between the development and operations department need to be reversed. Development needs to be measure on stability and operations needs to be measured on throughput. This gives development incentive to deliver production ready applications while encouraging operations to push applications into production.

Through automation, we can reduce the cost of making changes to the application and thus enable frequent delivery of software. This enables us to make smaller incremental changes that are less risky than big bang approaches. We want to create a deployment pipeline and automate the entire delivery process from build, deploy, test to release. Provisioning of infrastructure as well as database migrations and deployments need to be also automated so that we can always test in production like environment. All this creates transparency with a complete audit trail from when a change was introduced into source control until it was released into production.

Measurements are key to continuous improvement. We need measurements for all teams.
Are we generating business value? How many new users do we have? How many new orders?
Are we producing quality products? How many bugs are there? What changes are breaking things?
Do we have the right infrastructure? How often does a failure happen? How long to restore service?
From these different measures we can decide on what areas need improvement.

Techniques like CI, TDD, and refactoring that developers use to improve their code can also be used to improve the Ops environment. Having the development team sharing and working with the operations team to improve their environment further builds trust and improves the relationship between the two until gradually the border between the two disappears.

All this is a major improvement on the traditional silo in the organizations. By building trust, aligning incentives, building a deployment pipeline, and measuring our processes, we can now eliminate the hardening and stabilization phases that tend to extend project plans. As the 2 groups move closer together, done no longer means dev complete but instead means released and in use. Developers now write production ready code and deploy to production like environments from day one. The conception to production cycle gets dramatically reduced to a matter of days and the business can now receive fast feedback and plan based on real user data.

In my next post I’ll describe how this enables organizations to align themselves around products instead of projects. Stay tuned...

Adapted from Enterprise DevOps: Breaking Down the Barriers between Development and IT Operations @ADP East 2011

Sunday, April 29, 2012

At the ADP East conference, Jez Humble made the case for continuous delivery and DevOps.

He described a simplistic view of an organization where you’ll have a business department that comes up with a new product and hands-off requirements documents to the development department. After a couple of years, the development team finally completes the application and hands it over to the operations department so that they can provide the application to the customers.

This cycle from conception to production takes a couple of years and the success criteria are based on on time, on schedule and on budget. Very few organizations go back to analyze the total life cycle cost of the product vs. the real customer value it is generating. According to the Standish group, about 60% of features delivered end up being never or rarely used.
Since the business does not have a mechanism of delivering quickly and validating their ideas through early customer feedback, they instead try to forecast and predict the future by guessing what their customers will want. And since acquiring funding is hard and in order to ensure project success, they end up cramming the requirements documents with all sorts of features that cover all imaginable scenarios. This increases scope which in turn delays the project and further lengthens the conception to production cycle.

Agile addresses this problem by focusing on delivering working software early and often and avoiding the big batch death spiral approach to software development. However, even by adopting Agile, the organization needs to align the goals of these 3 departments. The business is being measured on innovation; development is being measured on throughput while operations are being measured on stability. These are competing goals that can impede Agile adoption as departments treat each other as rivals instead of partners.

Well what if these goals where reversed? What if development is being measured on producing stable, production ready applications while operations is being measured on throughput?
In my next post I’ll explore how Jez believes DevOps makes this happen by addressing culture, collaboration, automation and measurement. Stay tuned…

Adapted from Enterprise DevOps: Breaking Down the Barriers between Development and IT Operations @ADP East 2011

Tuesday, April 24, 2012

At the April DC Scrum User Group meeting, I presented the technical debt simulation game "Hard Choice". The game was developed by the Software Engineering Institute to help players understand the impacts of technical debt. Players need to balance investing effort to gain an advantage or paying a price to take shortcuts.

The game is a board game and the goal is to accumulate as many points as possible. Players accumulate points by finishing before others and by collecting tool cards. Along the way players can take shortcuts by crossing bridges. After crossing a bridge, players are penalized on each subsequent roll of the dice. To get rid of the penalty, players need to skip a turn. The games generated a lot of good discussion afterwards. Some of the comments were:

1. Taking debt late was better than taking it on early. But it got worse in the second round.
2. Not all debt was the same. Some bridges were better shortcuts than others
3. Knowing were others were on the board impacted the decision of whether to take a short cut or not
4. After taking a couple of short cuts we were barely moving
5. Not taking shortcuts made me loose!
6. I finished 1st but did not win

Overall, the game was fun and well designed. The discussion afterwards was very insightful.