Agile Software Development Hits Stride After Years of Evangelism

Agile Software Development Hits Stride After Years of Evangelism

The adoption of Agile software development practices is on the rise as enterprises have begun to see benefits from implementing Agile into their development shops.

However, the questions quickly become how much Agile actually pays off and even how much Agile is enough. The answers here are not so simple, as there is no one-size-fits-all approach to Agile. Essentially, as many practitioners will tell you, it all depends. It depends on the goals of the organization, as well as the size, culture and geography of the organization, among other things.

The Merriam-Webster dictionary definition of agile is "marked by ready ability to move with quick, easy grace" and "having a quick, resourceful and adaptable character." Agility is defined as "the quality or state of being agile."

Meanwhile, Wikipedia describes Agile software development this way: "Agile software development is a group of software development methods based on iterative and incremental development, where requirements and solutions evolve through collaboration between self-organizing, cross-functional teams. It promotes adaptive planning, evolutionary development and delivery, a time-boxed iterative approach, and encourages rapid and flexible response to change. It is a conceptual framework that promotes foreseen interactions throughout the development cycle. The Agile Manifesto introduced the term in 2001."

VersionOne, which is known for its Agile project management tools, recently released the results of the seventh annual "State of Agile" survey, which shows Agile adoption on the rise. Sponsored by VersionOne, the survey polled 4,048 developers between August 9, 2012, and November 1, 2012. Ninety percent of respondents said implementing Agile improved their ability to manage changing priorities.

Moreover, more people are also seeing value in terms of project visibility when implementing Agile—84 percent, compared with 77 percent in 2011.

The survey also showed that organizations are continuing to scale Agile beyond single teams and single projects. The 2012 survey saw a 15 percent jump in the number of respondents who work where there are at least five Agile teams, and a 9 percent increase in those working with up to five Agile projects. In addition, overall Agile momentum is up; the number of respondents who said they plan to implement Agile development in future projects increased from 59 percent in 2011 to 83 percent in 2012. Most are using Scrum or Scrum variants (72 percent), as in past years. Kanban and its variants nearly doubled in 2012.

Agile Software Development Hits Stride After Years of Evangelism

Scrum is perhaps the most dominant Agile methodology. Scrum is an iterative and incremental Agile software development framework for managing software projects and product or application development. Its focus is on a flexible, holistic product development strategy where a development team works as a unit to reach a common goal, as opposed to a traditional, sequential approach. Kanban is a method for developing software products and processes with an emphasis on just-in-time delivery while not overloading the software developers. In this approach, the process, from definition of a task to its delivery to the customer, is displayed for participants to see, and developers pull work from a queue.

For its part, Microsoft is an example of a company that has benefited from Agile as both a user and a producer. Aaron Bjork, principal program manager lead for cloud developer services at Microsoft, told eWEEK Microsoft's transformation to become more Agile began back when the company was building Visual Studio 2010 and Team Foundation Server (TFS) 2010.

"That was the first tool where we knew we had to start supporting Agile teams in a way we hadn't in the past," Bjork said.

At the time, Microsoft was on a two-and-a-half-year cadence for releasing its developer toolset. Visual Studio Team System (VSTS) 2010 introduced new features and capabilities to help Agile teams with planning. Preparation for that release began in 2008. And in the fall of 2010, Microsoft began its planning for Visual Studio 2012.

"We moved from six- to eight-week milestones to having three-week sprints," Bjork said. "Our entire division is aligned on this three-week cadence. We produce the Team Foundation Server as an on-premise solution, but we also provide the Team Foundation Service, which is a cloud-based offering. The services business is changing it for all of us. We realized we needed a different cadence and a different mindset to keep pace with the demands of providing a service. When you're an Agile team and you're working in iterations of two to three weeks, a goal is to have your product in a shippable status at all times. But the reality is unless you're shipping it, you're going to cheat. However, when you're delivering new features on a three-week cadence, you're not allowed to cheat."

Bjork said all of Microsoft is moving in this direction, but the Visual Studio team is a bit more in touch with Agile because of its close-knit relationship with developers, who request new features and support for Agile processes all the time, he said. "Agile, Scrum and Kanban were all just buzzwords for a time, but not anymore." Microsoft supports Scrum and Kanban in the Visual Studio and TFS tools.

"The place I see Agile paying off is that I'm able to engage with customers at a different level," Bjork said. "I can make changes faster, so I do a lot less apologizing for things we don't have than I had to do before. I can tell customers that something may not be coming in the next iteration, but I can get it in three months or six months down the road. We can meet the market where it is."

Agile Software Development Hits Stride After Years of Evangelism

As a user, Microsoft has one of the largest TFS installations in the world and one reason the company tweaked the tool was because it needed to bring it to a point where it could support the software giant's own needs, Bjork said. The starting point with this in the 2012 release was to focus on a typical Agile team in the company—a team of four or five developers, and a couple of quality assurance people—and address their needs.

However, before implementing any real Agile support in the toolset, Bjork said Microsoft developers trying to follow Agile practices used three primary tools: TFS, Excel and whiteboards with sticky notes. "It was so much that when we got started [building in Agile support], the first thing we built was a virtual task board to replace those sticky notes. We had a daily standup meeting around that board, and we wanted to bring that into the tool."

As evidence that Microsoft has gone all in with Agile, Bjork told eWEEK the Visual Studio team will soon be moving into a new building on Microsoft's campus that has no offices, just shared open spaces and team rooms that are conducive to Agile development.

Microsoft uses the traditional waterfall method of development as well as Scrum, other Agile variants and Kanban. The waterfall model is a sequential design process, often used in software development processes, in which progress is seen as flowing steadily downward—like a waterfall—through the phases of conception, initiation, analysis, design, construction, testing, production and maintenance.

"Traditional waterfall development methods meant unending project schedules and almost farcical rates of project failure," said Jake Sorofman, a Gartner analyst in a blog post entitled "Agile Isn't Just for Geeks Anymore." "Far more often than not, the deliverable at the end of a 12-month cycle wildly missed the original requirements—or the requirements had changed in the fullness of time."

"The Agile movement has arrived," Bjork said. "Agile is about ways to deliver value to your customers faster and better."

Joel Semeniuk, executive vice president of Telerik's Agile Project Management Division, told eWEEK, "The goal of Agile is to remove waste from what you are doing." This could mean something as simple as eliminating the need to redo certain tasks.

Telerik makes and sells Agile project management and testing tools. "The entire company is Agile-based," Semeniuk said. "We think, we live and we breathe Agile inside the company." However, Agile is not just a switch that can be turned on, he said. "Agile itself needs to be adaptive and incremental. … We support an Agile buffet table. We don't promote a shotgun approach to Agile."

This sentiment is shared by service providers and consultants that deliver Agile products and services for hire. Theo Schlossnagle, founder and CEO of OmniTI, an IT services company, said his team is versed in Agile practices and brings Agile to the table in its engagements.

However, he said, "We have a lot of projects going on, and we try not to shake the boat. If a client is using Agile, we go in and use what they have or we use the tools we are familiar with—like JIRA or the entire Atlassian suite, which can be used in Agile and waterfall projects. It's really about good project management more than anything else. You have to be able to be flexible."

Flexibility is essential in Agile development, said Michael Rosenbaum, founder and CEO of Baltimore-based Catalyst IT Services, which provides Agile onshore solutions for customers.

Rosenbaum told eWEEK, "You need to be able to use the principles of Agile but have flexibility. ... We are less ideological about methodology. If you can match with the culture of the client, you can decide the best sub-methodology for the situation you're dealing with."

Indeed, Rosenbaum said he typically sees two primary benefits to Agile development: IT alignment, or the alignment of technical teams with the goals of the business and enabling the technical teams to have a bigger role in the business, and innovation.

"Technology is increasingly at the core of what is driving enterprise business models, so there needs to be technical folks who can participate in the conversation of what the new revenue lines should be," he said.

Agile Software Development Hits Stride After Years of Evangelism

Catalyst is an Agile app services shop with development centers in Baltimore and in Portland, Ore. "We marry big data with team assembly," Rosenbaum said. "We collect large amounts of data and then build algorithms that find the right developers and put them on the right teams and then we put these teams with the right clients. It lets us match the culture of the team to the culture of the client. It also lets us deploy teams of up to 15 people anywhere in the U.S. within 24 hours. "

Meanwhile, back to the question of Agile payoff, Laszlo Szalvay, vice president of Scrum business at CollabNet, said it is difficult to quantify payoff "because you can look at it from a number of different angles, such as financially, employee retention or time-to-market. We're seeing a broad but shallow adoption of Agile globally."

Szalvay said one of CollabNet's major Agile success stories is helping Intel's product development engineering group implement Scrum. Scrum had four major impacts at Intel: reduced cycle time, the elimination of schedule slips and missed commitments, improved morale and increased transparency.

"Scrum has been a major contributor to a consistent, repeatable, 66 percent cycle time reduction in the creation of our work product," wrote Pat Elwer, a principal engineer at Intel, in a report on the Scrum project.

In another use case, Apartments.com used Serena Software tools to move to Agile. The company's technology team is divided into separate groups that support the consumer-facing Website. Each group typically does two-week sprints and has its own release plans and deployment schedule.

Apartments.com's move to Agile has progressed and has led to streamlined releases. But there's room for further advances, said Matt Stratton, director of technology operations for Apartments.com. "We want to get to a level of maturity in release management where the Agile development teams are responsible for releases and for all the steps leading up to a deployment, and the production operations teams take it from there," said Stratton.

Meanwhile, the Federal Home Loan Bank of Atlanta moved to Agile with VersionOne's tools. Before adopting Agile, FHLBank Atlanta was struggling with software quality and delayed releases, said Greg King, assistant director of IT software delivery at the bank. Since adopting Agile, the bank is now seeing a 92 percent reduction in open production defect tickets, on-time and within budget project delivery, and improved collaboration between software delivery and business units, among other benefits, King said.

Yet, despite proven benefits of Agile, barriers to its adoption exist, such as cultural barriers that prevent some teams from accepting the Agile way of life.

"This year's survey highlighted a major disconnect between the perceived level of executive knowledge about Agile and the importance of having executive-level buy-in for Agile to truly succeed," said Robert Holler, CEO and president of VersionOne, regarding the "State of Agile" survey.

Mik Kersten, CEO of Tasktop, told eWEEK that in spite of all that's good with Agile, there are two key problems with it. One is that Agile started as developer-centric, and is now being driven from the executive office.

The other is that "today's Agile deployments are focused on shortening iterations in development," Kersten said. "That works great for small teams and for many ISVs. It's less great when traditional requirements, quality management and large-scale operations are in the loop and running at very different cadences.

"To get the benefit of Agile in the large, we need to stop ignoring those disciplines, and to apply the concepts of Lean Manufacturing to the end-to-end software lifecycle," he continued. "Our Agile development practices need to evolve into "Lean ALM" [application lifecycle management] processes that connect each of the stakeholders of software delivery in an integrated build-measure-learn loop."