Using BGP RTBH in VPNv4

Document

This is a ‘how to’ type of article about implementation of the BGP RTBH feature in VPNv4 environment. We will briefly describe how this feature might be useful for an ISP and how to adapt it to the MPLS VPN service. Before reading further, it is advised to take a look on the RFC 5635, which describes the concepts behind it and on the Cisco's whitepaper, which details the implementation steps in IPv4 Unicast environment using Cisco IOS.

OVERVIEW:

In general, RTBH, which stands for Remotely Triggered Black Hole, is a BGP based mechanism to black-hole some traffic based on source or destination and thus there are two options exist for RTBH: Source based and Destination based. In this example we will focus on Source based implementation as it is more efficient than Destination based.

For an ISP offering MPLS VPN service this feature might be needed if it also offers a ‘Central Services VPN’, like VoIP Services, “Internet access via VRF” or “Overlapping VPNs” to let customers interact and collaborate. In other words, when the number of VPNs is big and connectivity requirements are complex, making number of possible threats significant.

Typically black-holes are what we definitely do not want to have in our BGP core as it means that some traffic will be lost or dropped and we have a set of techniques to prevent such type of issues. But under certain conditions, like DDOS(*) attacks coming into our core and subsequently destined to our customers,we might be interested to drop some traffic before it will flood vital circuits. There are plenty of solutions against such type of attacks, like for instance Cisco Traffic Anomaly Detector and third-party products.In this scenario we are not going to discuss them further, as we are interested in a built-in IOS capability to defend a network against a DDOS attack. To be clear, there is no such a feature like “RTBH capability” listed in the Feature Navigator. This technique is solely based on normal BGP operations.

One may recall the old-days CAR - Committed Access Rate approach, available via ‘rate-limit’ command under interface configuration mode and used to restrict amount of traffic coming in/out based on the configured limit (nowadays it is more or less substituted by set of MQC style commands). This is also a good and easy tool to protect network core and customers’ circuits against flood. The only thing missing here is…scalability. Imagine you have a dozen of border routers and hundreds of exit interfaces. Even assuming that you need to configure only external interfaces to drop unwanted traffic, it turns out to be a configuration overhead. The BGP RTBH way is much more automated compared to the CAR approach. However, since we do not use any access/prefix-lists for filtering in RTBH, there is no way to select which type of traffic to drop.

For a global view of the methodology how to mitigate DDOS attacks please review RFC 2827.

For basic configuration examples on IPv4 unicast please refer to the document specified in the beginning of this article.

In this case-study we are using VPNv4 as infrastructure.

LEGEND:

Let’s assume that we are running a big ISP under BGP AS 65023 and among others offering MPLS VPN service to our customers. The big ISP means lots of iBGP sessions (we don’t like synchronization ) and this is why we would usually divide a network into multiple confederations or deploy Route-Reflectors. In this case-study R2 acts as RR, R3 is a client of R2 for VPNv4 iBGP, and R1 belongs to a vrf TYCHO, using eBGP as a PE-CE protocol. Another possible implementation of eBGP in a VPNv4 environment and thus relevant to our setup might be the Option B for Inter-AS VPN deployment - Single-hop Multiprotocol eBGP for VPNv4.

On the diagram R3 has a vrf named BAUD, where malicious host(172.16.1.254/32) resides – the attacker. In Source based RTBH filtering for a given source will occur on every router which learnt the advertisement, no matter where exactly an attacker resides.

To generate an advertisement we normally use a special device called ‘trigger’ to send a BGP update to the rest of the network. It might be a separate BGP speaker or even a host running a BGP daemon. The only requirement is that the trigger should have BGP peerings with all the routers, or with a route-reflector in case one is deployed. In our case R2 will act as a trigger too.

So on R3 which is connected via iBGP the prefix is learnt, but not picked up as best, and on R1 there is no attacker's prefix at all.

To let R3 install the prefix we should tell it how to reach 192.0.2.1, i.e. we need a static route. Since we are dealing with vrf it would be logical to create a host route for 192.0.2.1 under vrf BAUD pointing to the Null0 interface to actually black-hole traffic, like this:

R3(config)#ip route vrf BAUD 192.0.2.1 255.255.255.255 null 0

But this will never allow R3 to install 172.16.1.254 into RIB:

R3(config)#do sh bgp vp u all

NetworkNext HopMetric LocPrf Weight Path

Route Distinguisher: 65123:123 (default for vrf BAUD)

*> 172.16.1.0/240.0.0.0032768 i

* i172.16.1.254/32192.0.2.102000 ?

Actually ‘debugbgp vpnv4 unicast’ shows following output:

2d06h: BGP(2): no valid path for 65123:123:172.16.1.254/32

EXPLANATION:

The reason why is because the next-hop resolution process for the VPNv4 routes will use the global routing table and the static route for 192.0.2.1/32 therefore needs to be configured in the global routing table:

The BGP session between R1 and R2 is eBGP, therefore uses directly connected interfaces for peerings. By default IOS checks that the next-hop of a received update via eBGP belongs to a subnet directly connected to the local router. Otherwise next-hop attributes for those sessions would be useless. In our case the route to 192.0.2.1 considered not directly connected because the next hop(which is Null0 interface) is not on the same subnet(10.0.12.0/24) as both peer addresses. In turn it fails default verification requirements.

To overcome this requirement we can use commonly used approach by allowing multiple hops between eBGP neighbors:

You might have noticed the command 'no ip unreachables' under interface Null0:

!

interface Null0

no ip unreachables

!

By default Cico IOS, as a good Internet citizen, will send an ICMP 3/1 message once it receives an unroutable packet. Which is 'should' and good habit in a friendly environment. However, since we are defendig against a DDOS attack, sending too much unreachables may overload a router itself. Apart from powerful CoPP, there is a built-in rate-limiter for ICMP type 3 messages with default of 1 message per 500 ms. It would be also a good idea to disable them for the Null0 interface, since it is mainly being used for intentional drops. Here is a good discussion about this command.

UPD:

Technically speaking, we have achieved the desired outcome and adapted BGP RTBH for VPNv4 prefixes, but there is another technique and RFC worth to mention here: RFC 3704 - Ingress Filtering for Multihomed Networks, which can complement Source-based RTBH.If enabled on autonomus system edge interfaces, it will check every incoming packet to verify whether it has corresponding route enry in the router FIB, and if not drop it. This effectively prevents spoofing attacks.

The command used to make it works is:

R1(config-if)# ip verify unicast source reachable-via any

In our setup we could enable it on R3's interface facing BAUD and on R1's interface facing BGP core, since DDOS supposed to come from there.

If you are interested to learn more about uRPF, please follow this link.

thanks for pointing this out, indeed I was primarily focusing on VPNv4 fixes and forgot to mention uRPF supplement. As for 1.200, it was a stale route left after some other tests. I have fixed both points.