Agile

02/24/2010

Recently, colleague Dave West and I published some research showing how Agile has gone mainstream in both the tech industry and IT departments. Here's another sign of the mainstreaming of Agile: Collabnet's acquisition of Danube.

If you follow the market for developer tools, you've seen milder indicators of this trend already. For example, last year, Atlassian acquired Greenhopper as a plug-in for JIRA. Other tools vendors, such as Compuware, have added Agile configurations to their existing products.

Collabnet's acquisition of Danube is a bigger step. Danube's core product, ScrumWorks, is a lot more than just a configuration or a plug-in. Collabnet now has a fully-featured PLM tool that's designed to model and support the Scrum process. Not only is Collabnet writing a big check to buy Danube, but it's also making a commitment in its portfolio strategy to fit ScrumWorks into its constellation of other products. That's not a trivial amount of additional engineering, support, marketing, and sales effort that Collabnet is assuming. (FYI, here's Dave West's take on this acquisition.)

Other developer tools vendors should ponder Collabnet's move. Moving from Waterfall to Agile changes more than just the checklist for completing a release cycle. Agile also requires changes in values, and even a completely different concept of what a release represents. To support this profound transformation, tools may need more than just tweaks to their functionality.

12/08/2009

Even though I see more than my fair share of PowerPoint slides on a daily basis, I still like skimming through SlideShare to see what people have posted. I liked this presentation by Si Alhir that tries to explain Agile and Lean to people who have no background in it.

However, this presentation wouldn't work for every audience. Sure, the presentation is as comprehensive as it could be, given the medium, but some people don't need a comprehensive introduction. Instead, they need something with a very spare amount of content that they can use to convince people back at the ranch that Agile and Lean are worth further investigation. (Lean, perhaps?)

This presentation gets closer to reaching that objective. It's not necessarily the "Agile For Lean For Dummies" presentation, but the author probably wrote it with a different audience in mind.

11/02/2009

I almost wish that this excellent post from Tyner Blain wasn't titled "Agile Prioritization: Which Widget?" If you left out the word Agile from the title, you'd still have a good cautionary tale about how you prioritize features and capabilities. More importantly, something phrased as a development project ("Build the following widgets...") doesn't necessarily convey what a feature does does, how it provides value, and why the team should build it in the first place.

On the other hand, to give Agile its props, user stories have moved the quality of requirements forward. By their nature, user stories force people to make at least some minimal effort towards explaining how someone will use a piece of technology. Depending on the quality of the user story, plus other supporting references (personas, etc.), the user story can explain rather easily why the technology has value.

Of course, you can write bad user stories, or fail to build the conceptual bridge between the human activity it describes and the technology that might support it. If everything in your backlog (or other requirements content, if you're not using that particular Agile medium) has titles like ERP widget, or even File upload widget, you're probably not building that bridge.

A very narrow definition of Agile has passed away, to be replaced by a
mature, expansive version that has now joined the mainstream of development
methodologies. Agile with a capital “A,” with its vision limited to the
development team, died of natural causes. Its successor still worries about
build scripts, daily Scrum meetings, and IDE plug-ins, but it recognizes the
sovereignty of business objectives, and governs jointly with other
methodologies. While we might talk more about agile with a small “a,” the
significance of this change is big.

You could see these signs of regime change throughout the conference. The
program book was a heavy tome, reflecting both the size of the audience for
Agile and the scope of questions that audience brought to Chicago. Instead of
presentations that argued the merits of one Agile approach over another, they
focused on topics such as scale, release management and executive sponsorship.
Serious project management issues such as managing resources and portfolios were
as popular as development processes, such as writing good stories or building a
continuous integration strategy.

Clearly, the breadth and depth of topics at Agile 2009 achieved a critical
mass of energy and enthusiasm, and then some. The number of sessions was
boggling, which made the conference program was a heavy tome. The sessions we
attended were full of people asking questions, sharing experiences, and
exchanging e-mail addresses so that they could continue these conversations.
Outside the official sessions, impromptu discussion groups and project
simulations filled ever alcove, seating and table area.

As apt as the
Julius Caesar reference was, a different Shakespearean allusion might
work even better. As you might tell from our description, Agile 2009 felt like a
significant point in the history of software development, the kind of moment
that reminded us of the famous speech from Henry V (paraphrased, with
apologies to the Bard):

We few, we happy few, we band of builders:For he today that finishes his
sprint with me,Shall be my brother: be he ne're so Waterfall,This day
shall improve his productivity.And product teams elsewhere, now
a-bed,Shall think themselves accursed they were not here;And hold their
projects cheap, while any speaks,That attended with us Agile 2009.

Looking at the data from numerous surveys (results to be published soon),
approximately 40-45% of teams are now using Agile in some capacity. But before
we crown some new Agile king, it really needs to show how it can re-shape the
entire software supply chain, determining how organizations define portfolios,
releases and IT strategies. This new model of governance, therefore, must apply
the principles of Agile to broader problems.

For this reason, before its coronation, Agile may need to assume a different
name, Lean. Where Agile has re-defined how people work within the boundaries of
the development team, Lean’s dominion is the larger organization. Lean builds
upon Agile, as well as other disciplines that apply to parts of the organization
that share their boundaries with the development team. You might say that Agile
governs one province, but Lean governs the entire kingdom.

Certainly the vendors at the conference have started to take notice of what
this regime change means. The vendor village contained the old nobility of Agile
tools—Rally, VersionOne, Danube, and others. As the boundaries of Agile/Lean
expand, other vendors, such as Microsoft, IBM, Serena, and Microfocus (including
their recent Borland acquisition) have joined the court. These vendors are less
provincial, but they have had to assimilate the local dialect and customs of
Agile. At the same time, the Agile tools vendors have been learning how to be
more cosmopolitan, without losing touch with their essential roots.

In the next year, rather than continuing to jostle and elbow each other, some
of these vendors may join forces as part of mergers and acquisitions. Others may
decide to focus just on the provinces they know best. For vendors and their
customers alike, the near future may be, to use (or abuse) Shakespeare again,
“the brightest heaven of invention.”

By the way, if you have access to Forrester's research, here's the study I did last year on this topic. As I said during the interview with Steve, I thought it would be a bit of a lark. Instead, it turned into a real puzzler, trying to find the organizational algorithms that technology companies used to develop a variety of different PO/PM formulas. They're the same! They're different! They're different levels of seniority! They're the same! And so on.

08/27/2009

During the Agile 2009 conference, we got news that a truck hit Ken Schwaber, one of the Agile movement's founders, while he was cycling near his home in Massachusetts. All the best to him for a speedy recovery.

08/26/2009

Next Monday, August 31, Forrester colleague Dave West and I will be presenting some of our preliminary results from our Agile adoption survey. (Which is still open, if you're interested in participating.) We'll be covering Agile adoption in both IT departments and technology industry companies, including the differences between the two.

Of course, everyone wants to know how many development teams have adopted Agile. My sneak peek for you is, "Substantially more than we had originally estimated." If you want to hear the full answer, here's the link for the teleconference.

08/14/2009

Forrester colleague Dave West and I will both be at Agile 2009 later this month. If you, Dear Reader, are also attending, I'd be delighted to meet you in person. Drop me a line to let me know that you're attending, and maybe we can figure out how to connect in Chicago.

Chances are that you have questions about Agile, the state of PM in the tech industry, social media, or other topics that I cover. If you're a Forrester analyst, let's turn the conversation into an official inquiry! I always prefer to have conversations in person than over the phone, and Agile 2009 may be a good opportunity to do so. Some sample topics from recent inquiries include:

How does the rest of our company need to adjust to Agile adoption in the development team?

How can we revamp our requirements process?

How can we use social media to help capture and assess good ideas?

How can we add product management discipline to our IT organization?

What's a reasonable estimate for the size of the R&D organization (including QA and PM) in a SaaS company?

08/07/2009

Hot on the heels of the requirements survey (which is still available, if you haven't participated), we've launched the Agile adoption survey. Colleague Dave West and I are looking into the state of Agile adoption, looking for answers to commonly-asked questions like...

What aspects of Agile are getting stronger or weaker adoption?

How important are tools, coaching, and other aids?

How important is the flavor of Agile you select?

What types of organizations or projects seem to have the best chances of success with Agile?

All responses stay strictly confidential. If you're wondering what's in it for you, we'll provide you with a copy of the research, once Dave and I have finished writing it up. And please feel free to forward the link to other people who might take the survey.

07/20/2009

Someone should make a poster out of the main points of this article, and sell it at Agile 2009. Only someone who has been around the software development block several times would recognize the sure-fire ways to undermine improvements to the development process like, Replace a plan document with a plan “in your head” that only you know.

In this episode, Israel Gat illuminates the ways in which Agile adoption depends on organizational and cultural factors. We also muse about Helmut von
Moltke, 19th century military Agilist. (Go look it up!)

Plus, a brief review of an even briefer document about innovation in “knowledge-creating” companies, and a heads-up about some upcoming survey research about product requirements and Agile adoption. Copyright (c) 2009 Tom Grant

06/18/2009

Many thanks to Israel Gat at The Agile Executive for posting my thoughts on how Agile is following the same path that many revolutions take. After you've had some initial successes, and take your new programme seriously, what now?

05/18/2009

Jeff McKenna's role in the history of Agile is as long as it is deep.
This week on the Heretech podcast, Jeff and I talk about Agile adoption in the technology industry, and why
Agile is much more than just a process. Copyright (c) 2009 Tom Grant

04/17/2009

While the actual document about Agile usage in the technology industry slowly but majestically navigates through the final editing and production process, I thought I'd share an important bit of data that didn't make it into the report. Shown in the diagram below is the percentage of survey participants who said that they used particular Agile methodologies. We also asked the respondents about other methodologies that often come up in discussions of Agile, either as complementary approaches (for example, Lean), or as points of comparison (Waterfall being the most obvious one).

How many at each table in the Agile seating chart?As you can tell, there's no clear winner here, other than Scrum, which overshadows every other Agile methodology in adoption. There's another way to look at the same picture: whenever there's a family photo of Agile methodologies, the participants almost always make sure to invite their poor cousins to attend. There are both good and bad sides to that earnestly ecumenical approach.

Someone once told me, "If you want to learn about Agile, read a book." Most of these references have an earnestly ecumenical approach to Agile. For example, Craig Larman's Agile And Iterative Development: A Manager's Guide gives equal time to Scrum, XP, UP, and Evo. Peter Schuh's Integrating Agile Development In The Real World covers a different set of methods, but with equal egalitarianism. Sure, these books are a few years old, but they're representative of what teams interested in Agile will find when they scan the bookshelves.

We don't need to declare a winner. A methodology that's in the minority may shoot forward tomorrow in adoption. Scrum might become so commoditized that it becomes the basis for a new set of tailored approaches, in which case the original meaning of "Scrum" winds up submerged in other things.

However, I can tell you from first-hand experience that many development teams are having a tough time figuring out if they'll find a healthy community of fellow adopters for particular Agile methods. That's a very legitimate question, particularly since the success of Agile implementations often turns on what lessons learned a team can apply from other people's experiences . (Which is one reason, perhaps, why Agile coaching seems to be as important as it is. Among other things, a coach is a storehouse of vicarious experiences with Agile implementations.)

Everyone is welcome at every tableActually, the survey results shown in the diagram tell a different story about methodological pluralism. In practice, teams mix and match different methodologies. The pluralism of methods within teams may be just as interesting, and maybe more interesting, than pluralism across teams. My Forrester colleagues Dave West, John Rymer, and Mike Gilpin wrote an excellent piece on using Lean to complement Agile, and vice-versa. The diagram shown here indicates that a lot of other methodologies (Six Sigma, ISO, etc.) occasionally wind up in the mix, too.

My upcoming research document, which covers a lot of other aspects of Agile adoption, provides some more information about this mix-and-match behavior some additional teams. The real topic of the study, however, is how Agile changes the way that groups interact with Development, and the effects these changes have on the company at large. I'll post when it becomes available. There's a second document in the works, too, which I'll talk about later.

03/02/2009

This interesting InfoWorld article makes a lot of useful observations about security issues in the Agile process. The way Agile practitioners feel their way through the design doesn't seem like a good way to approach some of the difficult up-front security requirements that extend across sprints. The biggest problems might be at the birth of an application, when many architectural decisions happen (or get postponed).

The article got me thinking a bit about building Agile teams to address security issues for an already-released product. New security threats require rapid responses, often requiring flexibility and creativity to handle. Work from this team will have to be folded into the core product, but then it becomes just another set of entries in the burndown list.

02/23/2009

Speaking of social media, one of the two research documents now in the editing queue looks at using social media as a source of product requirements. Using Forrester's POST* methodology as a starting point, how can product managers harness the enormous amount of potentially useful information transmitted in the clear through blogs, forums, Wikis, and similar technologies?

The other document in editing is the "Agile company" piece, covering the results of the survey and interviews we conducted to understand how Agile development changes technology companies. To foreshadow the results, I had to divide Agile adoption into two stages. To date, Agile aficianados have focused on the first, Agile within the development team. Clearly, for the story of Agile adoption, that's only Chapter One.

* In this approach, the steps for analyzing social media involve people, objectives, strategy, and technology (POST).

About the Heretech

Tom Grant is a senior analyst at Forrester Research. You can e-mail him at tgrant@forrester.com, or reach him via Twitter at TomGrantForr.
All opinions expressed here are my own, and not necessarily those of my employer, Forrester Research.