Blogs

Agility@Scale: Strategies for Scaling Agile Software Development

Scott is a Senior Consulting Partner of Scott W. Ambler and Associates, working with organizations around the world to help them to improve their software processes. He provides training, coaching, and mentoring in disciplined agile and lean strategies at both the project and organizational level. He is the founder of the Agile Modeling (AM), Agile Data (AD), Disciplined Agile Delivery (DAD), and Enterprise Unified Process (EUP) methodologies. Scott is the (co-)author of 19 books, including Disciplined Agile Delivery, Refactoring Databases, Agile Modeling, Agile Database Techniques, The Object Primer 3rd Edition, and The Enterprise Unified Process. Scott is a senior contributing editor with Dr. Dobb's Journal and his company home page is ScottWAmbler.com

Agility@Scale: Strategies for Scaling Agile Software Development

A common misunderstanding about agile software development approaches are that they're only applicable to small, co-located teams. Yes, it's much easier to be successful with small teams, and with co-located teams, and as a result agilists being smart people prefer to work this way. After all, why take on extra risk when you don't need to do so? But, sometimes reality gets in the way and you find yourself in a situation where you need a large team, or you need to distribute your team (see previous blog postings for strategies for distributed agile development), and you would still like to be as agile as possible. The good news is that it's still possible to be agile with a large team, although you will need to go beyond some of the popular "agile in the small" strategies to succeed.

Here are some disciplined agile strategies to succeed at large-team agile:

Question the need for a large team. Many times an organization will believe that they need a large team because their process is overly complex, because they're still organized for waterfall development, or simply because that's what they're used to. I've seen teams of 80 people doing the work of 20 as the result of over-specialization of job roles and all the bureaucracy required to organize and validate their work.

Do some initial envisioning. In order to succeed the team must work together towards the same goals. This is true for small teams but doubly true for larger ones -- without a common vision chaos will quickly ensue. You must gain this common vision on two fronts: you need a common business vision and a common technical vision. To gain the common business vision you must do some initial, high-level requirements envisioning and to gain the common technical vision some common architecture envisioning. This isn't to say that you need to take on the risk of detailed, up-front specifications but you must at least have a high-level understanding of the scope and technical solution in order to move forward effectively. So, expect to spend the first few weeks of your project doing this initial modeling.

Divide and conquer. You never have a team of 200 people, instead you have a collection of subteams that add up to 200 people. This is called having a team of teams.

Align team structure with architecture. The most effective way to organize the subteams is to have each one implement one or more components, and thereby to build your overall system as a "system of systems". This reduces the coordination required because the majority of the communication will be within the subteams themselves. You'll still need to coordinate the subteams, that will never go away, but you can reduce the overhead (and the risk) by being smart about the way that you organize the people. A common mistake is to organize around job function (e.g. having architects in Toronto, developers in Raleigh, testers in Bangalore, and so on). This increases communication overhead and risk because these people need to work together closely to get something built.

Project management coordination. Each subteam will have a team lead/coach, and these people will need to coordinate their work. There is often an overall project manager who leads this group. To coordinate the work within their subteam the team lead/coach will often have a daily meeting, in the Scrum method this is called a scrum meeting, where people share their current status and identify any problems they may be running into. To scale this effectively the team lead/coach attends a daily team coordination meeting, a scrum of scrums, where the same sort of information is shared at the overall team level.

Product owner coordination. Similarly, each subteam has a product ownder, also referred to as an "on-site customer", who is responsible for making decisions about the requirements and for providing information to the team in a timely manner. Sometimes a single product owner will work with several subteams. The product owners will get together at the beginning of the project to do some requirements envisioning to identify the initial scope and to start portioning the requirements between the subteams. Because the requirements between the subsystems are interrelated and should be reasonably consistent, the product owners will need to meet on a regular basis to share information, to negotiate priorities, and to resolve requirements-related disputes.

Architecture coordination. Each subteam will have an architecture owner, often a senior technical person and sometimes also in the role of the team lead/coach. These architecture owners will get together at the beginning of the project to do some initial architecture envisioning, based on the requirements envisioning efforts of the product owners. They will identify the major subsystems, and their interfaces, enabling the effective organization of the team into smaller subteams corresponding to the architecture. They will also get together regularly to evolve the architecture and to resolve any major technical issues.

System integration team. For complex systems, which is often what large teams work on, an effective system integration effort is critical to your success. Although this may be easy at first, as the overall system evolves the need for a subteam focused solely on this quickly becomes apparent. This not only supports the development efforts of the subteams, it also supports independent investigative testing.

Independent testing team. An independent testing team is common on mid-to-large size agile projects to enhance the testing efforts of the development subteams. This testing team will work in parallel to the developers, they get a new build on a regular basis (minimally each iteration, although more often is desirable), which they test in more advanced ways than what is typical with Test-Driven Development (TDD). For example, they often validate non-functional, quality of service (QoS) type requirements as well as technical constraints, things that often aren't captured well via user stories. They'll also do investigative testing to try to break the system by using it in ways not thought of by the product owners.

Some specialties reappear. On larger teams it can make sense to have some people be a bit more specialized than what we normally see on small agile teams. For example, it's common to see people in the role of agile DBA, tech writer, build master, or user experience (UE) professional. More complex systems often require people in these roles, although it still behooves these poeple to not be pure specialists but instead to be generalizing specialists with a wider range of skills. Also, recognize that the reintroduction of specialists can be a slippery slope back to the bureaucracy of traditional software development.