Next, the machine room had a broken chiller – this was so severe that the servers began to heat up almost immediately and we had to go into shutdown straight away whilst repair people worked on it. Services came back up the next morning, but only just in time for our training workshop.

Then came the big nasty DDOS attack against Janet, a higher education network which spans the UK. From the university, we could access some websites, but many more were down. Crucially, no one could reach our servers from the outside, even though this time they were running.

Apart from taking down HumanMine, FlyMine, and our websites, this also took down our CDN, meaning that anyone who hosted their own InterMine but relied upon us for their CDN was also affected by the outages.

Amazon Web Services to the rescue

We’re always working our hardest to make sure things run as smoothly as possible, but sometimes things are beyond our control. This number of outages is obviously less than ideal, and we’d like to do better in 2016.

One solution is to set up your own InterMine CDN, but could we do something to help you on our end, something that doesn’t require extra work for others? After a bit of thinking, we realised we probably could.

An InterMine registry

Another feature that’s been in planning works for a while is an InterMine registry. Like the CDN, it’s pretty important that a dynamic discoverable registry have near-perfect uptime. So hopefully you can expect to see the hardcodedmine lists disappearing as we start to take advantage of a hosting service with high availability.

Watch this space for the official announcement and release dates for the CDN upgrade and registry release!

A final note about outages

While email was okay when Janet was playing up, the machine room breakdowns took out our mail server, and hence the mailing list – so the “Hey guys, is your server down?” queries didn’t reach us until after everything had recovered.

If you’re aware of a problem, the best place to find out more will be our Twitter account, @intermineorg. We’ll always make an effort to update our twitter feed – by mobile network if necessary – with details of outages, whether planned or emergency.