Waterfall and the birth of agile: What your manager needs to know

If you began your software development career sometime after, say, 2008, and launched your first project using agile methods to code and deliver software, it’s possible you don’t know why many of your older colleagues hold agile principles so dear. Or why the Agile Manifesto itself was such a big deal when it went public in 2001, becoming part of the mainstream in software dev circles by the middle of that decade.

In those days, a genuine revolution was taking hold in software development. Old-school methodologists and traditional project managers found their entrenched practices under attack by upstart thinkers, who were setting aside time-honored “engineering discipline,” instead embracing what actually worked for managing software projects.

The upstarts had no beef with traditional engineering. Their argument was simply that it doesn’t work within the realm of software, which demands more creativity than scientific principles in meeting stated goals.

How software development developed

If you have team members or managers who need some background, TechBeacon Learn’s latest track, “Software Innovation for Business Leaders,” offers a quick review of what came before agile, DevOps, test-driven development, user stories, sprints, and all the other practices developers take for granted today within the landscape of software project management.

If you already know this, then forward the link to your senior manager, or anyone who may need a clearer understanding of concepts such as “stakeholder review” and “technical debt.”

Why was waterfall used in the first place?

A software methodology that insists on knowing all requirements up front and that requires design documents months before any coding begins may seem suspect, if not ludicrous, these days. But in the '70s and '80s, when the biggest money to be made in software was in the defense, civil engineering, and aerospace industries, planning was everything. It was how we waged and won military battles, how we built the Hoover Dam and jet planes, and how the federal government completed the interstate highway system. So why shouldn't careful, up-front planning be part of any software project management agenda?

The waterfall method assumed that stakeholder sign-off of a requirement or design document meant that the stakeholder would see a software system sometime later—even if that was months or years downstream—that accurately reflected all of those plans. But, as project after project proved, there were always too many moving parts, including the requirements themselves, for development teams to deliver satisfying results.

How waterfall pushed software over the cliff

TechBeacon contributor and agile expert Mark Balbes illustrates the problem in a true-to-life story involving all the teams that make up a traditional software project. These“tribes” in software projects have incompatible goals and timelines. Each serves different needs in the organization, which rarely understands their impact on software project outcomes.

Waterfall projects tended to fail due to an imbalance between a well-intentioned methodology and the typical outcome for large-scale software projects. You need to be able to describe at a practical level the traditional procedures used within the waterfall method, because you should know what software developers and managers who cut their teeth on projects in the 1980s and 1990s believed back then. Only then will you be able to fully explain today’s methods, including the ones you’re using on the projects for which some of those managers are responsible.

Tell your boss

You may know all of the reasons that agile methods gained acceptance in the early 21st century. But your immediate supervisor may not. In fact, supervisors may be responsible for so many things that your development methods may not come to their attention until a new, massive project comes along. That’s when they need to understand how you go about making your estimates while coping with their expectations.