Tuesday, December 16, 2014

How to block unwanted IP addresses on Linux efficiently

http://xmodulo.com/block-unwanted-ip-addresses-linux.html

You may want to block IP addresses on your Linux box under various
circumstances. For example, as an end user you may want to protect
yourself from known spyware or tracker IP addresses. Or when you are
running P2P software, you may want to filter out connections from
networks associated with anti-P2P activity. If you are a sysadmin, you
may want to ban access from spam IP addresses to your production mail
server. Or you may wish to block web server access from certain
countries for some reason. In many cases, however, your IP address block
list can grow quickly to tens of thousands of IP addresses or IP
address blocks. How can you deal with it?

Problems of Netfilter/IPtables

In Linux, banning an IP address can be done very easily with netfilter/iptables framework:

$ sudo iptables -A INPUT -s 1.1.1.1 -p TCP -j DROP

If you want to ban a whole IP address block, you can also do it as easily:

$ sudo iptables -A INPUT -s 1.1.2.0/24 -p TCP -j DROP

However, what if you have 1,000 independent IP addresses with no common CIDR prefix that you want to ban? You would have 1,000 iptables rules! Clearly this does not scale.

What are IP Sets?

That is when IP sets come in handy. IP sets
are a kernel feature which allows multiple (independent) IP addresses,
MAC addresses or even port numbers to be encoded and stored efficiently
within bitmap/hash kernel data structures. Once an IP set is created,
you can create an iptables rule which matches against the set.
You should immediately see the benefit of using IP sets, which is
that you can match against multiple IP addresses in an IP set by using a
singleiptables rule! You can construct IP sets using combinations of multiple IP addresses and port numbers, and can dynamically update iptables rules with IP sets without any performance impact.

Install IPset Tool on Linux

To create and manage IP sets, you need to use a userspace tool called ipset.
To install ipset on Debian, Ubuntu or Linux Mint:

$ sudo apt-get install ipset

To install ipset on Fedora or CentOS/RHEL 7:

$ sudo yum install ipset

Ban IP Addresses using IPset Command

Let me walk you through on how to use ipset command using simple examples.
First, let's create a new IP set named banthis (name can be arbitrary):

$ sudo ipset create banthis hash:net

The second argument (hash:net) in the above is required, and represents the type of a set being created. There are multiple types of IP sets. An IP set of hash:net type uses a hash to store multiple CIDR blocks. If you want to store individual IP addresses in a set, you can use hash:ip type instead.
Once you have created an IP set, you can check up on the set with:

$ sudo ipset list

This shows a list of available IP sets, along with detailed
information of each set including set membership. By default, each IP
set can contain up to 65536 elements (CIDR blocks in this case). You
can increase this limit by appending "maxelem N" option.

In the above, I tried removing an existing IP set using destroy option to see if I can restore the IP set.

Automate IP Address Banning

By now you should see how powerful the concept of IP sets is. Still
maintaining a up-to-date IP blacklist can be a cumbersome and
time-consuming process. In fact, there are free or paid services out
there which maintain these IP blacklists for you. As a bonus, let's see
how we can automatically translate available IP blacklists into IP
sets.
Let me grab free IP lists from iblocklist.com which publish various IP block lists for free or for a fee. Free versions are available in P2P format.
Here I am going to use an open-source python tool called iblocklist2ipset which converts P2P versions of iblocklist into IP sets.
First, you need to have pip installed (see this guideline to install pip).
Then install iblocklist2ipset as follows.

$ sudo pip install iblocklist2ipset

On some distros like Fedora, you may need to run:

$ sudo python-pip install iblocklist2ipset

Now go to iblocklist.com, and grab any P2P list URL (e.g., "level1" list).
Then paste the URL into the following command.

As of this writing, the "level1" block list contains more than
237,000 IP address blocks. You will see that that many IP address
blocks have been added to the IP set.
Finally, go ahead and create a single iptables rule to block them all!

Summary

In this tutorial, I demonstrated how you can block unwanted IP addresses using a powerful tool called ipset. Combine that with a third-party tool like iblocklist2ipset,
and you can easily streamline the process of maintaining your IP block
list. For those of you who are curious about the speed improvement of ipset, the figure below shows the benchmark result comparing iptables without and with ipset (credit to daemonkeeper.net).
Tell me how much you like it. :-)