Software Infrastructure

Software infrastructure is a lot like municipal infrastructure such as highways, subways, and stadiums. Everyone likes to enjoy the benefits but we seem to have difficulties on when to build it, what to build, or how to pay for it.

I like to think of software infrastructure as the software components you use that you aren’t in the core business of providing or selling. If your running a cloud hosting company then the virtualization and provisioning software are the product your selling. The software you use for processing credit card payments and emailing customers are infrastructure components you use. If your business is providing credit card payment services to websites then the software that does this is part of your core product but the virtualization software you use is part of your infrastructure.

Facilities for logging, message queues, automated deployment and configuration, monitoring builds and unit tests, tracking performance are all common examples of software infrastructure. This isn’t a post about build versus buy, this is is a post about figuring out when you need components, how good they need to be and how you should go about building and integrating them.

Infrastructure components like municipal infrastructure are usually used by different teams in your organization for different purposes. The individual communities along an expressway often can’t justify building a 6 line highway through their community but can justify it once they partner with neighboring towns to build a connected road. Software infrastructure works the same way. When multiple departments or business units start to connect with shared infrastructure benefits show up. Shared infrastructure components allow different departments to share data, expertise and costs.

However not all infrastructure will help accomplish this. One of my favourite episodes of The Simpsons is the monorail episode. The town of Springfield comes into some money and have a townhall to decide how to spend it. The townsfolk buy into a song and dance routine and build a mono-rail. The monorail is good for some laughs but not much else and it results in a trainwreck. This frequently happens with software where a slick sales demo promises a solution to cure all your problems convincing most people to go for snappy tunes instead of thinking things through.

So when do you build a piece of infrastructure? Many agile techniques including scrum say that you should only allocate resources for infrastructure items when you need them to meet your immediate business needs. The agile approach is to only build enough infrastructure to meet your needs in this sprint. You don’t pave the highway if the items in this sprint can get by with a dirt road.

One of the problems with this approach is that it gives other departments very little incentive to collaborate or share your infrastructure. It is easier for the team next door to build their own dirt road then to negotiate access rights and a route diversion on your dirt road. If you had built 4 lane paved road then it might be worth their while to work with you and help fund the ongoing maintenance but you didn’t build that paved road.

A good option is to work with the next town to upgrade the dirt road to a paved road when they show interest. With this approach you might build a lightweight logging infrastructure that meets your immediate needs. You let other departments know that you’ve done this and when they have some additional logging needs you invite them in to improve the logging infrastructure you wrote. Your neighbour turns your chicken-wire fence into a wood one to keep kids from wondering into his yard. This approach allows you to avoid expensive upfront infrastructure costs and temptations to build mono-rails while still being able to collaborate on shared infrastructure.

In the open-source world this happens all the time. One open-source project will submit patches ‘upstream’ to another open-source project to make the software more useful for themselves or to avoid having to duplicate something that another project has written. Open-source project developers tend to be good collabarators (otherwise they wouldn’t have an open-source project they will just have released their code under an open source license). The tools we use in open-source (git, public mailing lists with archives, open bug trackers, build instructions) all make this type of activity easier.

In the corporate world things are rarely this simple. Politics, juvenile ideas of code ownership, security policies, and cost accounting often come into play to discourage this type of collaboration. Centrally planned infrastructure components can avoid this problem, just like a good way to build a railway is to have the federal government drive the process. This might work if your upper management cares enough about a particular piece of infrastructure but, more often than not, this doesn’t happen because infrastructure components are not what your in the business of selling. Building centrally planned infrastructure is also not keeping with agile principals and requires a big up-front investment. A lot of the time agile development teams won’t know exactly what their requirements are for components far enough in advance to allow them to give meaningful requirements to a long term centrally planned project.

Another approach that can work is to have a team dedicated to building (or purchasing) software infrastructure components and working with other teams in integrating these components. This allows an infrastructure team to see the requirements of multiple teams and find common solutions. The infrastructure team would have a budget and mandate to build infrastructure. The challenge is to manage just-in-time demands from different teams in providing the components in a useable manner that meets the agile time-lines of the teams that need the infrastructure. A dedicated infrastructure team can also work on the infrastructure backlog by finding deficient infrastructure components that multiple products lack and then upgrading and integrating the improved components into multiple products.

Don’t settle for a spaghetti network of dirt roads or mono-rail train-wrecks. Learn about the infrastructure that other projects and departments in your company have built (or have bought/downloaded and integrated) and find a way that you can expand on it instead of building or buying your own. As your team builds components identify which ones fall into the infrastructure category and decouple them from your projects core code so that other teams can easily reuse these components when the need comes up.