4. Disable IPv6

5. Setup TCP Wrappers

This section will only be used if your server is accessed by a known list people from specific locations. If your server needs to be open to the public, skip this part.

Find out what IPs your ISP uses (you can use a site like http://ipchicken.com/ to find out your current IP). Do this on all the devices you will use to connect to this server (either via SSH, telnet, web browser, sftp, anything), including your phone.

Edit /etc/hosts.allow by adding a ALL: followed by the first set of digits of the IP you will use. For example, my file allows me to access from my ISP at home, by VPN and from my work.