When we started to work on the new release of the Content API, our team realised that it needed robust performance, high availability, and rapid scalability, because it was going be powering our partner's businesses as well as some of our core applications.

A search engine-based architecture was the obvious choice for the content API, given the feature set. But because the API proposition is that developers can sign up at will, and will be creating new applications that we haven't yet imagined - our ability to predict the load and nature of queries at any point is limited, compared with the dealing with load on guardian.co.uk, which our platform team are quite familiar with.

Firstly, therefore we needed to be able to rapidly scale the platform - if a great new application needs more power, we want to be able to say yes. So we're hosting the Content API in the cloud (yes, the cloud thing). This enables us to add and remove server power as required, without having to worry about over or under-committing costs. So, we needed software which was easy to manage as we scale up and down.

Secondly, as we plan to evolve the API, and will be introducing new content types and metadata in the future, we wanted to be able to make changes to the schema without having to worry about reindexing the content, or have downtime when we do this.

The technology we are using to power the API - Apache Solr, an open source search platform - works well with these requirements. As an open source product Solr/Lucene has been developed by a wide community, and is used by LinkedIn, Digg, AOL, MTV and many more. We really are genuinely impressed by how good this software is.

We were able to explain to the community that developed Solr how it was being used by the Guardian in its open strategy, and in particular how it was helping us to make the move from being a publisher to also being a platform. I've uploaded the slides on SlideShare and you can see them here:

In a separate session, Graham gave an in-depth explanation of how we implemented the Content API over the last few months. Network connectivity on the stage sabotaged the live demo, although many in the audience were able to explore for themselves at http://content.guardianapis.com once the wireless network came back up.

The API is a scala client of Solr using some simple tricks to index our entire body of content in just over an hour. We use Solr's excellent replication to quickly distribute this index to however many API servers we currently need, running in EC2. To ensure a smooth launch we fired up a few more servers for good measure. Here are Graham's slides from the event:

Guardian.co.uk is mostly powered by open source software, so it is great to meet some of the key people in the community of developers that contribute to it. We're overwhelmed by the creativity and vision in the conference sessions. We look forward to contributing ourselves in the near future, and plan to open source the code that delivers the Content API in the coming months.

We've enjoyed meeting up with some of the key people in the Solr/Lucene community, and would like to thank the conference organisers Lucid Imagination, who employ many of Solr's core committers. You can read more about our work with Lucid Imagination here.