Saturday, February 12, 2011

One of the most difficult components in any data center architecture to design and plan for is the access layer. In a traditional network hierarchy the access layer is where the most dynamic and changing requirements exist. Myriad technologies abound and can tell a history of the data center as new technologies were introduced with the progression from 100Mb Ethernet to 1G to 10G and the emergence of Unified Fabric (FCoE). Scaling these access layers has been a black art at times because of the changing pace of technology. What if you could have an access layer that meets your current 100/1G Ethernet needs today as well as 10G, provided a reduction in management points and helps tame the Spanning Tree beast? Enter the Nexus 7000 with support for Nexus 2000 Fabric Extenders (FEX).

The Nexus 7000s have been shipping for close to 3 years now and have a well established install base, mature software and have proven themselves as scalable Data Center platforms. The Nexus 2000 has been shipping for over 2 years and has been solving access layer challenges for customers very well when paired with the Nexus 5000 switch. Combining the two technologies provides similar benefits for the traditional FEX architectures only on a larger scale. Today the Nexus 5000 series support up to a maximum of 16 FEX while the Nexus 7000 supports 32 with current code and plans for more in the future. Let’s dig into the details.

First, what are the requirements for FEX support on the Nexus 7000? Three primary requirements must be met:1. NX-OS 5.1(1) or higher must be installed on the Nexus 70002. 32 port M1 10GE modules (part number)3. EPLD must be current to support VNTag

Once these requirements are met we can connect the FEX to the Nexus 7000. The options supported include traditional 10G Short Reach (SR), 10G Long Reach (LR) optics and Fabric Extender Transceiver (FET) for the M1 32 port card. The M1 32 “L” card add support for active Twinax cables which currently are available in 7 and 10M lengths. In our example, we’ll be using SR optics.

Let’s start by verifying we meet the requirements.We see below we are running NX-OS 5.1(2) so we’re good to go there.cmhlab-dc2-sw2-agg1# show verCisco Nexus Operating System (NX-OS) SoftwareTAC support: http://www.cisco.com/tacDocuments: http://www.cisco.com/en/US/products/ps9372/tsd_products_support_series_home.htmlCopyright (c) 2002-2010, Cisco Systems, Inc. All rights reserved.The copyrights to certain works contained in this software areowned by other third parties and used and distributed underlicense. Certain components of this software are licensed underthe GNU General Public License (GPL) version 2.0 or the GNULesser General Public License (LGPL) Version 2.1. A copy of eachsuch license is available athttp://www.opensource.org/licenses/gpl-2.0.php andhttp://www.opensource.org/licenses/lgpl-2.1.php

Now let’s check the EPLD*NOTE* This must be done from the default VDC and if an EPLD upgrades is required, it is disruptive so plan accordingly.cmhlab-dc2-sw2-otv1# install all epld bootflash:n7000-s1-epld.5.1.1.img

So we’re in good shape there, too. It’s like I’ve done this before….. :)Now that we’re ready, we’ve cabled the FEX to the switch via port e3/1-4 and we’ll be creating a topology that looks like this.

First, we need to install the FEX feature set. This is a bit different than what we’ve done with features in the past and must be done from the default VDC.cmhlab-dc2-sw2-otv1# show run | i fexcmhlab-dc2-sw2-otv1# confi tEnter configuration commands, one per line. End with CNTL/Z.cmhlab-dc2-sw2-otv1(config)# install feature-set fexcmhlab-dc2-sw2-otv1(config)# show run | i fexinstall feature-set fexallow feature-set fexallow feature-set fexallow feature-set fexallow feature-set fexcmhlab-dc2-sw2-otv1(config)#

Note that each VDC now has a config for allow feature-set fex.Next, we’ll go to our VDC where we want the FEX configured and get it setup.cmhlab-dc2-sw2-agg1# confiEnter configuration commands, one per line. End with CNTL/Z.cmhlab-dc2-sw2-agg1(config)# feature-set fex

Then we’ll define the FEX and specify the model. While this isn’t required because the FEX will identify itself to the Nexus switch, I think it makes the config more readable and is somewhat self documenting.

Note that today we cannot have a FEX multi-homed into two Nexus 7000s like we can on the Nexus 5000. Look for that capability in a future release along with support for additional FEX platforms.

When you think of the scale – 32 FEX x 48 ports = 1,536, that’s pretty impressive. Being able to take advantage of the cable savings with localized, in –rack cabling without the challenges of increased STP diameter, the FEX and Nexus 7000 make a powerful impact on the data center topology.

VIA,Thanks for reading the blog! Fast answer to all of your questions is that these capabilities are coming in the future. More specific information on when they will available can be obtained from your Cisco account team or partner engineer.

Post a Comment

Newest LiveLesson!

vExpert 2017

About Me

Field Engineer at VMware focused on NSX though blog posts are all my own. Husband, father, F1 fanatic and geek.
Ron Fuller is a Staff Engineer in the Network and Security Business Unit (NSBU) focused on NSX for VMware. He has 22 years of experience in the industry and has held certifications from VMware, Novell, HP, Microsoft, ISC2, SNIA, and Cisco including two CCIEs No. 5851 (Routing and Switching/Storage Networking). His focus is working with customers to address their challenges with comprehensive end-to-end Data Center architectures and how they can best utilize VMware technology to their advantage. He is the co-author of the VMware Press NSX Fundamentals LiveLesson video series. This adds to his existing body of work with CiscoPress. He has had the opportunity to speak in Europe, Australia and the United States on multiple networking and security topics. He lives in Ohio with his wife and four wonderful children and enjoys travel and auto racing. He can be found on Twitter @ccie5851.