Under the Covers with OTV

So, lets take a closer at OTV and how it works. As a reminder, OTV is an NX-OS feature that allows us to extend Ethernet LANs between data centers. One of the nice things about OTV is that it is transport agnostic–the connectivity between data centers can be L2 based, L3 based, IP switched or label switch–pretty much anything that can transport IP.

OTV works by creating an OTV control plane through authenticated links between the Nexus 7000 switches at each of your data centers (called edge nodes in OTV parlance). You can then “route” your LAN traffic by encapsulating it and routing it through this IP infrastructure. Routing of the traffic is determined by associating a MAC address with a next-hop IP address. The process is fully dynamic, so there is no need to establish and manage tunnels and virtual wires. This approach certainly simplifies management and administration over existing approaches, but it also allows you to take full advantage of your IP core such as optimal routing and features such as load balancing, multicast traffic replication, and fast failover.

The tables themselves are built transparently in the background, once OTV is configured, by proactively advertising MAC reachability. If the IP core supports multicast, the edge device advertises the address along with some extended attributes in a single updated that reaches all neighbors. The additional information, including VLAN ID, site ID, and associated IP addresses makes OTV a significantly smarter solution that other approaches, giving it the ability to support loop-free multi-homing, load balancing, First Hop Resiliency Protocol and ARP containment without generating increased complexity, processing overhead or the need for ancillary protocols. If the core does not support multicast, OTV can run with a “adjacency server mode” where one of the edge nodes is responsible for collecting and disseminating reachabilty information via unicast traffic. In either case, OTV’s approach is an improvement over other approaches that that require explicit configuration of each connection of depend upon flooding information.

As noted above, the extra information OTV uses makes is a much more intelligent connectivity solution. For example, OTV provides improved L2 fault isolation between locations: Spanning Tree BDPU are not forwarded into the core and neither are unknown unicasts. Cross site ARP traffic is reduced via proxy ARP capabilities in the edge nodes and broadcast traffic in general can be controlled via rate limiting and white lists.

So, what does all this technical goodness net you (yes, pun intended):

One-touch add/drop of sites without reconfiguration of other sites (point-to-cloud model)

Embedded intelligence obviates need for ancillary protocols such as VPLS

Dynamic design avoids the exponential complexity of virtual wires and the overhead and risk of flooding

When multihoming sites, OTV supports multiple active links on multiple edge (up to 16 per site) devices while avoiding loops for effective use of bandwidth and increased resiliency. OTV also supports both vPC and TRILL in this scenario as well as any multipathing offered by the underlying IP transport

Hi Omar,
How can we deploy OTV between 2 sites that are DC and DR? Currently, there are MPLS and dark fiber links between the sites for L3 and L2 links.
I am trying to understand how I can deploy the Nexus 7000 (currently not in the network) for OTV?

Hi Omar, I have two dark fibers, and I want to connect one Nexus in each data center,(P2P) my question is, is there any problems regarding the multicast requeriments in this scenario?.
Other thing, May I use the same dark fibers for the OTV, that is the join interfaces, and to route traffic between the two data centers at the same time???
Thanks.

Thanks Omar-How is it that we can implement the Jumbo frames and not affect the non jumbo traffic? Additionally, I read that the other switching platforms will also support the OTV, including the 6500 series specifically. What do you suppose is the time horizon for this addition to the feature set?Thanks!!

Has anybody worked with OTV and firewalls between the layer 3 networks? We need to comply with PCI requirements but also want to use OTV to connect us to a second/DR data center. Any insight would be appreciated.Thanks!!!Mike.

So, since OTV doesn't fragment (which is reasonable), it sounds like for this to be useful with regular old 1500 byte IP MTU traffic, one needs to ensure that the IP network between the OTV edges supports jumbos.I'm assuming that the recommended and preferred solution is to enable jumbos between data centers.Where that isn't possible, especially in the short term, is there something like an ip tcp adjust-mss"" option with it, or is this something we'd have to address on the end hosts?How many bytes does the OTV wrapper add?Thanks, David"

Mike:OTV will work with firewalls; however, since the traffic is encapsulated, the firewall will only be able to apply policy against the outer header, which may mirror some of the info that is in the encapsulated header, such as VLAN ID or QoS marking. However, many of the things a firewall is probably interested in are hidden from the firewall. The trick is to deploy your firewall outside of the OTV edges, not between them.Omar

Fabian:Thanks for your comment. I think enterprises will be please, but I also think service providers will find a lot to like about OTV and their ability to deliver servers.As to your question, OTV does not fragment frames. Forwarding is done in hardware and traffic is never directed to the CPU, relieving the CPU from any risk of overload.Omar

Omar, how is OTV supported through appliances that like to do things to IP traffic like firewalls, VPN's, and the like? Has Cisco done any testing with firewalls, what the next protocol in the IP header?

Hi Omar,Very nice feature for Enterprises.Until now I was using GRE with Bridging but had some trouble with frame fragmentation (CPU overloaded have to go to Nexus).Will I get the same issue with OTV?Thanks, Fabian.

Some of the individuals posting to this site, including the moderators, work for Cisco Systems. Opinions expressed here and in any corresponding comments are the personal opinions of the original authors, not of Cisco. The content is provided for informational purposes only and is not meant to be an endorsement or representation by Cisco or any other party. This site is available to the public. No information you consider confidential should be posted to this site. By posting you agree to be solely responsible for the content of all information you contribute, link to, or otherwise upload to the Website and release Cisco from any liability related to your use of the Website. You also grant to Cisco a worldwide, perpetual, irrevocable, royalty-free and fully-paid, transferable (including rights to sublicense) right to exercise all copyright, publicity, and moral rights with respect to any original content you provide. The comments are moderated. Comments will appear as soon as they are approved by the moderator.