I have a Debian server with KVM available 24/7. It was recently broken into and its root password changed.

My hosting provider has restored my access and helped me change the password. But in the general case, how can I protect myself from hacking and make sure this doesn't happen again? Also, how can I regain access to the server itself?

3 Answers
3

Although there's a lot that you can do to protect your box, but I do believe it is very hard to make sure this doesn't happen again if the server is not managed properly.

Well, with your root password being changed, literally your server is owned and it becomes a shared entity between you and the hacker which is a no no for sysadmins.

On regaining access to your server for a linux box(yours should be either a vps, dedicated server or your own server colocated at the hosting provider, but I may be wrong), I do believe that the usual answer is ssh? If you cannot access the ssh server with your credentials, then you may need to contact your hosting provider to help you on it.

After that, you need to find out the entry point on how the hacker penetrated your server(which is where the vulnerability lies).

Vulnerability at the processes that your server is running(apache, ftp, mail server, ssh)

Etc(not listed as it is too broad)

Howto deal with it:

Trace your server logs for any signs of hacking incident(if the hacker has already removed the logs then you're out of luck that you may need to check the codes on your own) and patch the vulnerability.

You can do a check up the here for any 0day vulnerability by typing the name of the software and the version of it and wait for the vendor to patch it up or you can patch it yourself(if you know how to).

Etc(not covered as there are too many solutions for different set of problems)

If you managed to retrieve a backup from your hosting provider during the hacking incident, it is advised that you don't restore it and use it as it is(just retrieve the content) as a shell may be embedded into your code and the hacker can just access it and hack it again.

Next time, disable the processes that you don't need(i.e mail server, ldap, or ftp if you don't send emails or you use a webbased service to upload your files) as it increases the possibility of being hacked(more attack vectors can be tried upon your server) or limit it to only your IP(get yourself a static IP in which you achieve through a few methods).

Whether to nuke your server or not, its your call although I do recommend you to nuke it.

If an individual could get enough control of your server that he could change the root password, then he basically owns the machine and keeps on owning it even if you changed the password back:

The vulnerability through which the intruder came in still exists until you find it and fix it, so he can come back at will.

The first task any half-witted intruder performs upon entering a machine is to plant a few extra accesses in case the sysadmin tries to get away with a few simple cleaning procedures on the machine. This is called a backdoor. People with root access can plant backdoors which you will never find from the machine itself (because the backdoor includes a kernel modification whose job is to hide the backdoor).

Therefore the only sane thing to do with your machine is to do what is traditionally called in these parts "nuke it from orbit". You are due for a reformat + reinstall from scratch. Sorry. Tough luck. For your next install, try to choose stronger passwords and to apply security updates more often (you should aim at daily updates, to minimize risks of compromise).

At least, the key advantage of IT over Medicine is that in the field of IT, you do not get sued for reformatting patients.

As it stands, the first part of the question (about how to protect yourself from being hacked) is probably too broad for a Q/A format. Securing a server in general is a very broad topic which there is an entire industry based around. Certainly keeping patches up to date and limiting the access to the server to the most restrictive access possible is a good start, but there is much more to it than that.

As far as how to regain access, as long as no data is encrypted, it is generally a simple matter of booting the hardware with another OS such as a LiveCD and then replacing the files that define the user with a known password. This does however not grant access to any encryption keys associated with the former user. If you don't have physical access to the box, then your options are far more limited and far more challenging and there isn't a one size fits all solution.

It's also worth noting that if your system has been compromised, simply changing the password won't fix the fact your system was compromised in the first place. Unless you simply had an incredibly weak password (and maybe even then), whatever they used to gain access and whatever they placed on your system while they had access may still allow them to regain control again. The best bet is to rebuild the system after backing up any critical data.