Vaultaire: a data vault for system metrics, backed onto Ceph

Before you can do serious analytics you need to have serious data. You can't go averaging it away just to save space; you need the raw data.

Vaultaire is a distributed metrics store. It is fault tolerant, performs well, is space efficient, and — since it stores data directly in Ceph — scales horizontally as your cluster grows.

Most systems crumble when faced with the thousands of metrics per second typically being gathered in a production environment. Ceph is good at absorbing lots of writes, and is especially good at handling randomly distributed reads.

You may have experienced Ceph via it's S3-compatible HTTP gateway or via the RBD block device service. You may not know these are built atop a simple library that handles talking to the storage cluster and locating your data, librados, and that's how Vaultaire stores its time-series data.

Using librados directly opens some pretty cool possibilities. It's powerful, but low-level, and as you'd expect there are gotchas. We'll talk about building a librados-based application, lessons learned relying on Ceph in production, and the challenges of building predictive modelling tools on top of a metrics store that doesn't throw data away.

Andrew Cowie

Somewhat unusually for a free software hacker, Andrew Cowie was an infantry officer in the Canadian army, having graduated from Royal Military College with a degree in engineering physics. He saw service across North America and a peacekeeping tour in Bosnia. He later ran operations for a new media company in Manhattan and was a part of recovering the firm after the Sept 11 attacks. He regularly consults on crisis resolution, change management, robust architectures, and (more interestingly) leveraging Open Source to achieve these ends. Andrew currently looks after the engineering group at Anchor Systems, where lately he's been working on scalable distributed systems, building server side code in Haskell and doing as little JavaScript as possible.