Blogs

Agility@Scale: Strategies for Scaling Agile Software Development

Scott is a Senior Consulting Partner of Scott W. Ambler and Associates, working with organizations around the world to help them to improve their software processes. He provides training, coaching, and mentoring in disciplined agile and lean strategies at both the project and organizational level. He is the founder of the Agile Modeling (AM), Agile Data (AD), Disciplined Agile Delivery (DAD), and Enterprise Unified Process (EUP) methodologies. Scott is the (co-)author of 19 books, including Disciplined Agile Delivery, Refactoring Databases, Agile Modeling, Agile Database Techniques, The Object Primer 3rd Edition, and The Enterprise Unified Process. Scott is a senior contributing editor with Dr. Dobb's Journal and his company home page is ScottWAmbler.com

Again and again I've seen IT organizations suffering from what I call the "Bureaucracy is Discipline" antipattern. For example, filling out forms and reviewing documents are both bureaucratic activities, neither of which require significant skill nor discipline to accomplish. However, agile practices such as developing potentially shippable software every iteration is easy to say but requires great discipline to accomplish. Respecting the decisions of your stakeholders, particularly those pertaining to requirements prioritization, is easy to talk about but proves to require great discipline in practice (particularly when you don't agree with a decision). It's easy to talk about taking a test-driven approach to development, but in practice it requires significant skill and discipline to actually do.

A "process smell" which indicates that your organization is suffering from this antipattern is a focus on following repeatable processes instead of focusing on repeatable results. An example of repeatable processes is following the same route to work every day regardless of driving conditions. An example of repeatable results is getting to work on time every day, but being willing to change your route as required, bicycling into work instead of driving, taking public transit, and so on. Nobody really cares how you get to work each day (the process), what they really care about is that you got to work on time (the result). Sadly, we've been told for decades now that repeatable processes are critical to our success in IT, yet when you step back and think about that's really a reflection of a bureaucratic approach. On the other hand, a focus on repeatable results is a reflection of a more disciplined approach. Interestingly, the DDJ 2008 Process Framework survey found that given the choice that people would much rather have repeatable results over repeatable processes when it comes to IT.

Mistaking bureaucracy for discipline, or rigour if you prefer that term, is a reflection of the cultural damage that has occurred over the years in IT organizations as the result of traditional philosophies and techniques. Unfortunately, this mistaken belief is a significant inhibitor to software process improvement (SPI) efforts, in particular agile adoption efforts, which must be addressed if you're to be successful. Overcoming this challenge will require a significant cultural shift in some organizations, and many people (particularly the bureaucrats) will find this uncomfortable.

An inhibitor that I run into again and again within organizations that are still in the process of adopting agile development techniques is something that I call the "We're Special" anti-pattern. The people involved believe that their situation is special, that some unique factor in their environment makes it all but impossible to adopt agile techniques, and therefore they need to continue to work in the manner that they've always worked, regardless of the obvious inefficiencies of doing so.

An organization suffers from this agile adoption anti-pattern when they start giving domain-based or technology-based excuses for why they can't become more agile. For example, I've heard bank employees claim that "Agile works fine for building web sites, but we're building financial systems therefore agile won't work for us", telecom employees claim "Agile works fine for building financial systems, but we're building embedded systems therefore agile won't work for us", and government employees claim "Agile works fine for embedded systems, but we're building web sites therefore agile won't work for us." Needless to say I often struggle to not roll my eyes.

The reality is that the business domain that you're working in doesn't dictate your ability to adopt agile strategies. I've seen very successful agile projects in banks, insurance companies, manufacturing companies, retail companies, pharmacueticals (yes, life critical applications), telecoms, and government agencies. I've also met people working in those domains claim that they're special because of the inherent challenges of the domain.

Similarly, technology isn't an issue. I've seen project teams that were successful at applying agile approaches using Java, VB, COBOL, C, Linux, Windows, System Z, on PCs, and so on. Granted, some technology platforms suffer from a plethora of "agile tooling", PL/1 comes to mind and I'm sure that there's a few more niche platforms where this is the case, but with a little online searching it's often possible to find good open source tools out there (or what's stopping you from starting such a project?).

The primary issues around agile adoption are cultural in nature. Can you become more flexible in your thinking? Can you become more disciplined (agile requires greater levels of discipline than traditional approaches)? Can you build a collaborative environment with your business stakeholders? Can you move away from bureaucratic processes to ones which focus on adding real value? Can you invest in your IT staff to give them modern development skills required for test-driven development (TDD), continuous integration, and agile database techniques (to name a few)? Addressing the "people issues", the cultural issues, is the hard part of moving towards agile.

If you're looking for valid excuses for why your organization can't move to agile, here's some valid adoption inhibitors that I see in organizations all the time:

Our project management office (PMO) has been trained and certified in traditional strategies and struggles to come to grips with agile project management techniques

We don't have the funding to train, educate, and mentor people in agile techniques

Middle management is threatened by agile strategies because their role clearly needs to change

Senior technical staff, in particular our architects, don't accept the need to roll up their sleeves and be actively involved on project teams

Our IT governance effort is not itself being governed effectively and is all but out of control, focusing on bureaucracy instead of enabling development teams to succeed

Our data management group insists on working in a serial and documentation heavy manner

Our QA/testing group insists on detailed requirement specifications

Our staff is overly specialized, resulting in numerous hand-offs between analysts, developers, architects, testers, and so on

The above list is just the tip of the iceberg. The point is that the real problems that you face are cultural in nature, not domain-based nor technology-based. It is possible to overcome these inhibitors to success, but you need to understand that you're suffering from them, what the implications are, and how to overcome them. This is one of the facets of the Health Assessment portion of our new Measured Capability Improvement Framework (MCIF) service, the goal of which is to help organizations improve their internal IT processes. Although MCIF isn't specifically about becoming more agile, the reality is that there are a lot of great agile practices out there, and some of them are good options for your organization. Assuming of course you get over your misconception that you're special for some reason and instead accept the need that you've got some hard slogging ahead of you to improve your IT game.