Agility@Scale: Strategies for Scaling Agile Software Development

Scott is a Senior Consulting Partner of Scott W. Ambler and Associates, working with organizations around the world to help them to improve their software processes. He provides training, coaching, and mentoring in disciplined agile and lean strategies at both the project and organizational level. He is the founder of the Agile Modeling (AM), Agile Data (AD), Disciplined Agile Delivery (DAD), and Enterprise Unified Process (EUP) methodologies. Scott is the (co-)author of 19 books, including Disciplined Agile Delivery, Refactoring Databases, Agile Modeling, Agile Database Techniques, The Object Primer 3rd Edition, and The Enterprise Unified Process. Scott is a senior contributing editor with Dr. Dobb's Journal and his company home page is ScottWAmbler.com

Agility@Scale: Strategies for Scaling Agile Software Development

When it comes to testing on agile projects it is common practice for agile teams to adopt a "whole team testing" approach where the team itself does its own testing. To accomplish this agile teams will often embed testers in the development team. Programmers will work closely with the testers, often via non-solo development strategies such as pair programming, to pick up their valuable testing skills. The testers will in turn pick up new skills from the programmers, and in effect both groups will move away from being just specialists (testers or programmers) to being what's called generalizing specialists. Whole team testing can be very different from traditional approaches where programmers may do some testing, often unit testing of their own code, and then throw it over the wall to testers and quality assurance (QA) professionals for verification and validation.

The types of testing that the parallel independent test team performs may include:

Pre-production system integration testing. Does the solution work within your overall organizational ecosystem? Importantly, if this is one of several teams currently developing new solutions, does this team's solution work with what will be in production (including the work in progress of other teams) when they go to release? In mid-to-large organizations the only economical way to do this sort of testing is via an independent, centralized team.

Usability testing. Although it's possible to do usability testing on the development team, the reality is that usability testing is a specialized skill that few people have (although could pick up via non-solo development). Furthermore, particularly for solutions with many potential users, you may want to invest in a usability testing lab. This is a centralized resource, or an outsourced resource these days, which is shared across many teams.

Security testing. Security testing is also a specialized skill, albeit one well supported with sophisticated security testing tools such as the Rational Appscan suite which can be included in your continuous integration (CI) strategy. Many organizations will centralize their security testing efforts.

Exploratory testing. The fundamental goal of exploratory testing is to discover where the solution breaks, as opposed to confirmatory testing which focuses on showing that the solution conforms to the requirements (this is the type of testing the development team typically focuses on). Exploratory testing is also a skill, a good one which everyone should strive to pick up, but exploratory testers are often few in number in many organizations. So, to leverage their skills effectively you may want to have some of them on the independent test team while they mentor others while doing so.

Non-functional testing. Non-functional requirements have a tendency to fall through the cracks on some development teams. Knowing this the independent test team will often "test to the risk" and focus on non-functional issues.

And much more. The above points are just exemplars, not an exact list. Please follow some of the links above for greater detail.

I'd like to leave you with several important thoughts:

The developers still do the majority of the testing. Just because there's an independent test team it doesn't imply that they are the ones doing all the testing. In fact, nothing could be further from the truth. They should be doing the minority of the testing effort, albeit the more difficult forms of it.

An independent test team will support multiple dev teams. For example, a test team of 5-6 people could support several development teams totalling 70 to 80 people. I typically look for a 15:1 or 20:1 ratio of developers to independent testers, hopefully even higher than that.

You need to consider better tooling. Although the development team will still be using common agile testing tools such as the xUnit and FIT frameworks the independent test team (ITT) will need more sophisticated tooling. First, the ITT will need to be able to report defects back to the team easily. When the development team is using a Jazz-based tool such as Rational Team Concert (RTC) then this can easily be done using either RTC (the web interface may be sufficient) or another Jazz-enabled product such as Rational Quality Manager (RQM). Second, the ITT will likely need more sophisticated testing tools, such as Rational Appscan for static and dynamic security testing and Rational Performance Tester (RPT) for performance testing (just two of several software quality management tools you should consider).

Independent testing is economical. Although I listed several tools in my previous point (hey, I do work for a vendor after all) an "unfortunate" implication of my advice (unfortunate for IBM at least) is that you can reduce the number of licenses that you require and still get this critical testing done by centralizing their use.

It may be a bit more complicated in regulatory environments. In a strict regulatory environment the independent test team may need to repeat, or at least validate, the testing efforts of the development team. In regulatory environments my fundamental advice is always this -- Have practical people, including yourself, read and interpret the regulations. If you leave it to the bureaucrats you'll get a bureaucratic solution.

This is an important scaling technique. Parallel independent testing, when done in an agile manner, is an important technique which you should consider when scaling agile strategies to meet the uniques needs of the situation that you find yourself in.

Yesterday I was involved with a workshop around agile development at scale. At one point in the conversation we started talking about the relationship between cost and quality. Some of the people in the workshop were relatively new to agile and still believed the traditional theory that to build in high quality it costs more, sometimes substantially more. This does appear to be true on traditional waterfall projects, but some people were making the mistake that this was an "natural law of IT" which also must apply to agile project teams. I naturally jumped on that idea and described how agile developers have found that writing high quality code leads to lower development costs and shorter time to value, in direct contradiction to traditional theory. A few people struggled with the idea for a bit, and one was pretty adamant that in some cases the need for very high quality does in fact lead to greater cost and time. He talked about his experiences on large-scale Rational Unified Process(RUP) projects and in particular how some URPS (usability, reliability, performance, and supportability) requirements can increase your cost. At this point Per Kroll, co-author of Agility and Discipline Made Easy: Practices from OpenUP and RUP, jumped into the conversation and pointed out although higher quality does lead to lower cost in most cases, using Toyota's lean approach to manufacturing as an example, that the agile community didn't completely have the relationship between quality and cost completely correct. My spidey sense told me that a learning opportunity was coming my way.