Farm Credit Services of America doesn’t sound like
an organization that courts controversy. The cooperative
association makes loans to more than 66,000 Midwest farmers and
cattle ranchers so that they can buy cows and pigs and tractors
and backhoes. Its main reason for existence—providing $11
billion of operating capital and real estate financing to those
who feed America—is as homey as the images of corn
fields, gently rolling green pastures and rugged, resolute
farmers that adorn its marketing materials.

It’s also based in Omaha, Neb., known more for steaks
than as an avant-garde laboratory for one of IT’s most
hotly debated development methodologies: agile programming.

But agile is exactly what Farm Credit Services has embraced,
whole hog.

Agile programming means different things to different people,
but at the core of all agile development methodologies are
these principles: Business stakeholders are colocated with
small, autonomous development teams; the teams rely less on
up-front requirements and documentation than on face-to-face
conversations; those conversations provide a continuous
dialogue for software design, testing and refocusing. The
constant refocusing, its advocates say, leads to more timely
and useful business tools. (For a tutorial on agile
programming, see “ABC: An Introduction
to Agile Programming”.)

Agile’s ascendancy is in direct response to IT’s
dolorous history of software project failure, cost overruns and
the concomitant business dissatisfaction with traditional IT
design and development—the waterfall methodology—in
which development slowly cascades through a series of steps
including requirements analysis, design, implementation,
testing, integration and maintenance. But for a variety of
reasons, not everyone has warmed to agile. In fact, just 17
percent of North American and European enterprises use agile
development processes, according to Forrester Research’s
“Enterprise Agile Adoption in 2006” survey.

Farm Credit Services welcomed agile programming because the
waterfall method had been failing the organization, as it has
many others. “We got requirements and would build [the
applications], and nobody was happy at the end,” says
Farm Credit Services CIO Dave Martin. One particular project,
which was a migration from a mainframe-based customer
application-processing system to a Web-based version called
PinPoint, involved more than 200 pages of requirements and, by
the end of 2004, had taken nearly three years to complete. In
the interim, the requirements and business needs had changed,
and most of the members of the original business team were
gone. The resulting bug-filled system was shelved not long
after its shaky debut.

And that’s not unusual. According to the 2006
“State of Agile Development” survey by The Agile
Alliance and Version­One, respondents said only 29 percent
of traditional projects were “somewhat successful”
or “very successful.” (Conversely, respondents said
81 percent of their agile projects achieved that level of
success.)

The Standish Group, which famously compiles its Chaos data
on software project failures, reported in its 2006 research
that just 16 percent of waterfall projects succeeded as opposed
to 41 percent of agile projects. Standish Group Chairman Jim
Johnson, who has been studying project failures for years, says
it “boggles” his mind why companies still resist
agile development. “To say that companies or CIOs are
reluctant to embrace agile is like saying they wouldn’t
take aspirin for a headache,” he says. “And
they’re not only not taking the aspirin, they’re
banging their heads against the wall and wondering why it
hurts.”

Going Agile

Martin decided to “shake things up” two years ago.
After some gentle nudging from Lou Thomas and Beth Schmidt, his
directors of applications development, the team adopted Scrum,
which has two-week iterations (called sprints), daily meetings,
and frequent iteration reviews and testing. “Our premise
is to have potentially shippable product every two
weeks,” says Thomas.

At Farm Credit, there are six development teams composed of
a business analyst, project leader, lead developer, two or
three developers, a database engineer, one to three business
owner participants and a QA engineer. In place of reams of
requirements, development teams write “user
stories” as the project progresses, detailing business
functionality enhancements and technology, successes and
challenges. “User stories are the main mechanism to
convey the business needs,” Martin says. Farm
Credit’s waterfall projects used to average 100 or so
defects per rollout; agile ones now average zero to two.

“We’ve rolled out five key products with
phenomenal results,” Martin says. “In each case our
business owners are ecstatic with the end result.”

Agile, Agile Everywhere? Well, No.

CIOs of companies large and small say they have either
abandoned waterfall methodologies or are gradually phasing them
out. “I don’t do the ‘I’ll deliver
everything to you in 18 months,’” says Raymond
Dury, CIO of Fifth Third Bancorp in Cincinnati, who has adopted
an agile method called Extreme Programming. “That’s
long gone.”

But waterfall processes are not, in fact, long gone. In
addition to the aforementioned Forrester survey (just 17
percent using agile), a March 2006 survey that polled readers
of Software Development magazine and Dr. Dobb’s Journal
found that 60 percent were not using any agile methodologies in
their organizations at all.

Perhaps not coincidentally, CIOs are actively looking for
project management assistance. Almost three-quarters of CIO
readers are either “extremely interested” or
“very interested” in finding out how to improve
their project management discipline, according to our latest
survey.

All this leads to one obvious question: If agile development
is so darn good, then why hasn’t it been universally
adopted?

The Trouble With Agile

The ceremonies of software development are deeply ingrained in
IT. With traditional waterfall processes, the business throws
its requirements over the wall to developers who hole up and
start coding as they see fit. An 18-month target date can seem
like decades away. Lost afternoons are no big deal. Who cares
what the “lusers” (coders’ derogatory term
for users) really want?

“A lot of people on the IT side thought [agile] was
the flavor of the month,” Martin recalls. “Some
just said, ‘I’m not going to do it.’”
(Those programmers, Martin says, have been churned in
agile’s wake.)

Opposition to agile methods also can come from enterprise
architects, project managers and quality assurance staffers,
says Carey Schwaber, a senior analyst for application
development at Forrester Research. Enterprise architects worry
that there’s not enough up-front design with agile, and
the consequence is spaghetti code. Agile teams are
self-managing, and the project leader’s role shifts
dramatically—from ordering around to facilitating. And
since QA testing happens throughout the process, and not just
at the end, there’s usually resistance from the testing
folk.

Misapprehensions about agile still run rampant in IT
organizations. Eugene Nizker, a former financial services CIO
and current consultant, ticks off the most infamous ones: Agile
teams do not plan. Agile teams skip design. Agile teams do not
test. Agile means no documentation.

In addition, executives can feel left out of the daily
scrums and sprints of agile life, engendering insecurity at top
levels. All this has hindered agile’s acceptance, says
John Scumniotales, one of the creators of Scrum.
“It’s easy to talk about the value of building
software this way, but if I’m betting my enterprise on
this project, senior management needs some controls and
visibility into the process,” he says, citing the need
for an agile-specific tool that functions like a Gantt chart,
which visually illustrates project progress.
“That’s where we need to get to,” he
says.

But given the epic floods of waterfall failures, CIOs owe it
to themselves and their organizations to get agile. “The
world just doesn’t hold still and wait for 18 months
anymore,” Fifth Third Bancorp’s Dury says.

Case Study: Quick to Market

Dury hasn’t abandoned traditional development processes
at his company, which has almost $100 billion in assets. Some
development projects, he says, such as those that affect base
infrastructure or intensive SOA projects, just
“can’t handle the type of intense change”
that comes with agile methodologies. But he’s excited
about his team’s adoption of Extreme Programming.

The head of the bank’s retail product line told Dury
about a new product the bank wanted to sell, an investment tool
(which would later be called Stock Power CD) that would offer
customers stock market–based earnings potential on their
certificates of deposit. In truth, the bank was playing
catch-up. “We were probably missing the market [for this
type of product],” Dury says, “and we wanted to be
in the market.” So speed was key.

Using Extreme Programming tactics, his team talked through
solutions that would meet the business needs quickly and be
technically feasible. More conversations ensued with retail
banking (lots of face-to-face time, little up-front
documentation). Two development teams, working in concert with
each other and the consumer banking and retail line
departments, formed within IT: the investment adviser team and
the CD development team. The teams worked closely with the
bank’s affiliates (which would eventually sell Stock
Power CD) and investment advisers to satisfy legal and
financial requirements.

Extreme Programming—daily huddles and continuous
business input, as well as recurrent
testing—“created the urgency to provide this
service to the customers ASAP,” Dury says. Just four
weeks after the initial conversation, several of Fifth
Third’s affiliates started offering the Stock Power CD
product. “Normally a new deposit product introduction
would take about 90 to 120 days,” Dury notes, adding that
the new product created “significant additional
deposits.”

Dury says he couldn’t imagine doing this project any
other way. Along with the faster time to market, it also
enabled “a closer working relationship with the business
side” and enhanced the business’s confidence
“in our ability to deliver and meet
expectations.”

The Alignment Add

As Martin’s and Dury’s stories show, agile project
methods demand continuous dialogue between the business and IT.
Colocating the business and IT team members engenders a kinship
that’s usually absent in traditional software
development.

“They’re on the ride with you,” says Ajay
Waghray, Verizon Wireless’s CIO of the Midwest U.S. area.
“If something is not going right, they’re
partnering with you right there.”

CIOs who have adopted agile methods say that changing their
relationships with business stakeholders can have some
surprising results. Waghray recalls recent conversations with
key operations and marketing executives in which he used a more
agile approach. “I would tell them on the call, ‘I
don’t want any requirements. What do you want? Just tell
me, and we’ll talk about it,’” he says. The
executives asked if they should write down what they wanted.
“And I would say, ‘No, no, no. Don’t give me
requirements.’” The approach, he says, “was
foreign to people.”

Verizon Wireless’s VZ Navigator product, which
provides users with turn-by-turn, voice-supported directions on
their mobile devices and has become one of the company’s
most popular mobile services, was heavily agile-influenced.
Waghray’s team consisted of stakeholders from IT, sales,
customer care, marketing, training and finance. The
team’s main focus, Waghray says, was to develop the
“simplest complete” solution right away, and then
build upon that to make better solutions. “The reason it
was successful was that the [agile] methodology encouraged
constant communication and alignment across all stakeholders,
leading to parallel development of not only the IT solution but
also of training documentation, test cases, external and
internal interface validation and development, and user
communication to get them ready for launch,” he says.