Tonight I am going to be moving to new servers. All the Ozblogistan sites (Andrew Norton, Catallaxy Files, Club Troppo, Henry’s Lore, John Quiggin, Larvatus Prodeo, Sensible Social Policy and Skepticlawyer) will be offline during the move.

I am making this move because performance has been steadily worsening in the past few months. My current provider, Linode, have more powerful plans available. However our problem is not that we need more RAM or more powerful virtual CPUs. Our performance is constrained mostly by disk performance.

Our new home, DigitalOcean, build their systems using much faster disks than Linode currently do. I also considered hosting with Australian provider OrionVM, but Australian bandwidth is simply too expensive. Absurdly so, compared to everywhere else.

Starting after 7.30pm Western Australian time, Ozblogistan will be down, possibly for several hours. Hopefully you all have better things to do on a Friday night than mash the F5 key.

Approximately 4 hours ago, Ozblogistan began to receive a very high level of traffic. During this period my performance tracking tools stopped working so it is difficult to tell how much traffic was received during the period.

A few minutes ago I kicked over our web server to reset all connections, which resolved this matter last time. This time too, apparently.

There are three possibilities:

Misconfiguration of the web server software

A badly behaved indexer program

A denial of service attack

I am confident that the server is correctly configured, but I cannot eliminate the possibility that I am misinterpreting the available performance records. That kicking the server over “fixes” the problem might suggest this possibility.

If there was a badly behaved indexing program crawling all Ozblogistan sites, it should have turned up in my Cloudflare dashboard which, so far, it has not.

The odd man out in the Ozblogistan network is Catallaxy Files. The Cat is the only site currently not “behind” Cloudflare’s protective service. If we were receiving the loving ministrations of a badly behaved indexing program, then that program was exclusively crawling the Cat.

The unhappiest possibility is that this was a denial-of-service attack; again, directed against the Cat. Sometime in the next few days I expect that Catallaxy Files will be placed behind Cloudflare along with the other Ozblogistan sites and that this will prevent any such attacks from being successful.

Ozblogistan was moved to new servers on Monday night by some subcontractors. The ensuing few days have been a mess because the move was only incompletely successful. In particular, bloggers have been unable to log into the wordpress backend to write posts or moderate comments.

This morning a threw a switch and it appears as though we are slowly coming back online. I’m very sorry for the disruption.

Once the move is finalised, I expect that Ozblogistan will be snappier than it is now. I also won’t have to worry about fixing security problems, doing backups, finding out why plugin X has stopped working, working out why the site has suddenly died and so on. These will now be responsibilities I can delegate to the providers.

This has been in the works for some time. While I was a student I could afford the time it takes to keep Ozblogistan working relatively smoothly. As a professional I no longer have that time. So it makes sense to hire other professionals to do it for me.

The security problem we had a few weeks ago (and which is still causing false positives) was the straw that broke the camel’s back.

Moving providers is never pretty. Things are going to break this weekend. And various things will probably appear broken into the new week. It’s all for the good. Ozblogistan will shine out ever brighter.

Update Saturday 2:17PM WST: Failure. I will probably have to try again on Monday night. I’ve re-enabled comments.

This morning I received two independent reports of trojan warnings being given for two different Ozblogistan websites.

After investigation, I have determined that the server was automatically compromised, presumably by a brand new attack (since we just 2 days ago updated to WordPress 3.4.1), and a trojan inserted into various parts of WordPress.

I have identified and replaced the affected files with clean copies, and you should see no more warnings.