In the Autumn of last year we took the plunge and started to implement agile as our technique of choice for creating large web sites. We had no idea what we were doing. Nine months later, we’re only slightly less clueless but are starting to get a glimpse into the true potential that an agile approach offers projects and workflows.

I think agile is one of those concepts that’s either easy to misinterpret or difficult to embrace fully, particularly in an industry that’s often so fixated on defining every aspect of a project upfront. This extends right from the sales process into specifications, contracts and design and, being the OCD uptight control freak that I am, I know I certainly struggled with the idea of breaking away from these rigid comforts. However, now that we’ve spent quite a long time working with agile – and I daresay, experimenting with it (sorry, clients!) – I find myself wanting to utilise it as an approach more and more, particularly on the really sizeable jobs (more on that later).

Of course, it’s not all been clear sailing and we’ve certainly had our fair share of struggles mainly because, in hindsight, we tried to only partial implement an agile approach instead of adopting it fully. As I’m sure any pro would testify, you need to stick to your guns and enforce agile principles completely or you just end up in some strange halfway house of project management that doesn’t benefit anyone, neither the developers nor the client.

Still, after attending conferences, reading and absorbing all of the resources we can get our hands on, learning from masters like Jim Shore and Zsolt Fabok, and, of course, applying it in practice, we’re slowly starting to get to grips with agile. Although it’s not for every web project, especially the smaller ones, it is looking like an incredible potent methodology.

Agile in a nutshell

You’ve probably guessed that I’m no agile samurai yet (so read this entire article with that great big dirty caveat in mind) but I’ll do my best to convey the principles of it as best I can. I’ll also throw in some useful references at the bottom of the post for anyone who wants to learn more from some, uh, professionals on the subject.

To be honest, the general idea is pretty simple and rotates around three core notions: it’s impossible to gather all the requirements at the start of a project; whatever requirements you gather will change; there’s always more to do than time or money will allow. Pretty logical, all-in-all, and I’m guessing if you work in the web industry (or any service industry), you’ll be nodding along vigorously right about now.

So with these ‘truths’ in mind, agile treats budget and timescales as two fixed parameters and that the only varying component in a project is the functionality that can be accomplished, which, y’know, makes a lot of sense. Instead of trying to define everything in a spec upfront (because you don’t know exactly what can be done yet or, indeed, how things will even look in four months time), features are simply brainstormed as user stories, assigned difficulty ratings and then prioritised. Nothing more. You then start work.

“There’s no way to force someone to be energized. However, you can remove roadblocks.”

James Shore

Although it can vary depending on how you practice agile, most of the time work is broken down into weekly iterations with the team tackling a set of features in each of these sprints. The point of this is to treat each iteration as an isolated piece of work, making it as self-contained as possible, accepting that the work you do will inevitably have to change as a new user story is tackled in a future iteration. With this in mind, you’ll generally work through features swiftly, making quick alterations to the project as necessary and deploying to either staging or production environments rapidly enough to ensure continual, hand-ons, informed feedback from the product owner.

Likewise, whilst some people advocate that to practice agile you need nothing more than a whiteboard and a pen, processes can become more sophisticated and structured by introducing scrum masters, daily stand-ups, design spikes and software like Pivotal Tracker (highly recommended). There are also lots of different implementation of agile, like Scrum and Extreme Programming, that vary and alter the approach somewhat.

Of course, it’s fair to say that there’s a lot more to it all than my rudimentary and, no doubt incomplete, summary above but hopefully it gives you the general idea about how it’s all meant to work.

Our difficulties with agile

The challenges we’ve had with agile vary. We’ve had trouble with everything from understanding how it exactly works, to how to project manage it effectively and even how to best integrate the content and design process into short agile iterations. The latter was particularly troublesome as design is best done from a holistic approach and, as Espen keeps spouting about, it’s almost impossible to design small subsets of features in isolation from each other (the solution we eventually discovered is to implement design spikes).

“Agile is like Churchhill’s democracy, the worst possible solution until compared to the alternatives.”

Perhaps the biggest challenge we’ve faced adopting agile though is in switching to the hugely different mind set it requires. It’s jolly hard to shift your brain from the idea of speccing features up front and then going through an entire design process that requires client sign off before you even start coding. However, as scary and difficult a hurdle this mindset is to overcome, if you succeed, you suddenly unshackle yourself from the rigid constraints such traditional processes apply. Strangely enough, it also fits in nicely with the recent responsive design movement and this crazy notion that you should actually include clients as much as possible in the work you do. After all, why shouldn’t they have the opportunity to see, comment, take part and change their mind about designs features before the final product is complete?

Another tricky issue can be obtaining client buy in, although, now that we’ve learnt how to manage the whole process somewhat better, I have to say that it’s actually not as hard as we initially feared. Again, it just boils down to switching out of the disposition of “fixed costs/fixed features” and getting clients to do the same, selling them the tangible benefits of agile instead. You also probably need to ask the question that if someone doesn’t agree with an agile approach, are they really the type of company you want to work with?

The benefits of agile

I don’t think it’s particularly difficult to appreciate the benefits of an agile methodology, at least not from a developers point of view. Unlike the notion of fixing everything up front – time, budget, functionality and number of designs – before you’ve even won the job and really know what the hell you’re doing, agile gives the opportunity to make important decisions as they arise. Not only does it make managing the workflow somewhat easier, it also allows the entire project to easily alter its parameters in response to either changes in the business model or user feedback. After all, just because something’s been written in a spec six months ago, doesn’t mean it will actually be relevant, useful, beneficial or even feasible when you come to implement it.

Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.

This flexibility also makes agile incredibly apt for web development as, arguably, no technology changes faster than the Internet. New products, techniques, technologies, styles and businesses pop up on an almost daily basis and working with agile gives both developers and product owners the best possible chance to adapt. What if a new, better, cheaper payment system arises half way through a project? Would it make any sense to stick to what was agreed several months ago just because it was written on a piece of paper (or go through numerous change orders and respeccing and recostings)? Or what if a complete new payment style, such as freemium, becomes the popular – wouldn’t the entire project be better off pivoting the business model to adapt to it instead?

Ultimately, agile lets you focus on delivering a product rapidly, within set boundaries of time and budget, ensuring that every iteration sees tangible and valuable features being developed and released. The flexibility such as an approach provides, especially in fast-paced web world, is incredibly valuable.

Size matters

Perhaps the biggest key difference in applying agile methodologies to web projects as opposed to traditional software management is often the scale of the work being undertaken. If you’re just building a ‘standard’ web site that can be easily scoped, comprising of, say, 10-15 days of work, how exactly do you apply an agile methodology to it? How can you break down two or three weeks into meaningful sprints? How do you even sell agile on a such a small scale to clients? For us, the answer has been that you don’t.

It’s something that we’ve battled with tremendously and have yet to find a perfect solution for (so we’re open to suggestions) as, to be perfectly honest, it just doesn’t seem very practical. Chances are you may well have completed the entire project in one or two iterations, thus rendering the whole point of agile slightly superfluous. Maybe there’s merit in opting to do very short two or three day sprints but, honestly, I find that notion a little laborious.

Of course, the core philosophy of agile can still help tremendously on projects of any size. Instead of going through a fixed design process, why not embrace flexibility, involve clients more, and engage with them during the creation process. Don’t show them ‘final’ visuals followed by a pixel-perfect build no matter what problems arose during it, instead show them work as it’s happening and openly discuss the issues being faced or the new ideas you’ve had along the way. It’s been our experience that people respond well to this true form of collaboration.

Whilst we’re still experimenting with agile development on smaller sites and may yet find the right way of implementing it, right now I do think it only really shines on large scale systems. When you start getting into the 50, 75, 100 days plus of work on projects that span months, the adoption of agile becomes a real, tangible benefit not just in development but also in project management and business results. The bigger the project, the harder it becomes to scope, manage and anticipate and those are the key fundamentals that agile truly helps with.

Does it benefit clients?

Yes! But then if I didn’t believe that, I wouldn’t be writing this article now would I?

I think the biggest concern clients have of utilising agile is that they aren’t getting a guarantee of receiving every component they could conceive during the tender or proposal stage of a project. It’s a fair point and, as someone who’s bought services from businesses before, I can appreciate the strong appeal of a fixed up-front quote that covers every eventuality. In reality though, it’s just not very realistic.

Deep down, we all know that it’s impossible to define exactly what features will exist or how they will be delivered at the start of a project and trying to shoe horn them all into a fixed fee and timescale is always going to boil down to the management of scope. Traditionally, if a supplier knows they only have X days to complete a task, they will devise a solution that fits that timeframe and reduce the scope of the design or functionality accordingly. It’s called project management and this containment of scope is ultimately no different if you’re working to fixed specs or agile workflows.

The real truth, let’s face it, is that clients want suppliers to be responsible for sucking up project overruns. They want developers to commit to creating a feature at the start of a project for a fixed fee and then implement it come hell or high water. Whilst some service companies do do this (it seems particularly prevalent in the UK and something we’ve fallen victim to many a time), most will fight over scope, re-cost the features anyway, or suck it up and grow resentful, producing a product that they can’t wait to be shot off.

“To say that companies or CIOs are reluctant to embrace agile is like saying they wouldn’t take aspirin for a headache….And they’re not only not taking the aspirin, they’re banging their heads against the wall and wondering why it hurts.”

Jim Johnson, Standish Group Chairman

Alternatively, agencies will win a project based on an estimated proposal price but then proceed through a charged specification phase which re-costs the job before contracts are signed. It’s a sensible approach and an alternative to agile although I would argue that it’s slightly disingenuous in that it leads to low-balling tenders with costs that aren’t ever going to be accurate. Plus, it still doesn’t provide any flexibility during the actual creation of the product itself.

Agile on the other hand is a truly open and transparent system of working, one that still gives the client the secure knowledge of fixed charges and fixed timescales by simply adjusting the breadth of features that can be accomplished. So whilst the product owner might not get all of the lower priority features they wished for at that the project’s conception, they are still guaranteed a viable, usable, and working product for no extra or hidden charges. Likewise, less of the job’s budget is spent on project management, change orders and scope limitation so more money goes straight to where its needed: design, development and content.

Through agile, clients obtain more flexibility and involvement during the project’s lifecycle and the ability to prioritise and alter features to adapt to changes in both their business and the web industry as a whole. And as someone who recognises how difficult it can be to not only create but then actually market and sell an online service or product, I think this is an incredibly powerful benefit.

Ultimately, any sort of business relationship boils down to trust. No matter what approach you use, no matter how tightly you try and specify or contract something, there always has to be an element of mutual collaboration (a key principle of the agile manifesto, in fact). Without trust no project will ever succeed and all agile development does is recognise this fact up front and implements a flexible solution to everyone’s mutual advantage.

Go use it.

Resources

Here are the some of the resources that have helped us come to learn and love agile development.