An Introduction to the Nexus 7700

We’re halfway through 2013 and we have our second new member of the Nexus family of switches for the year: the Nexus 7700. Here are the highlights:

Modular, chassis-based system

18 slot (16 IO modules) and 10 slot (8 IO modules)

True front-to-back airflow

New fabric modules

(6) fabric modules (maximum) per chassis

220G per slot per fabric module

1.32Tbps per IO module slot

Supports F2E and newly announced F3 IO modules

The F3 IO modules which were announced along with the 7700 are built to take advantage of the increased fabric bandwidth in these switches and also to shrink the feature gap between the F and M series of modules.

12x and 24x 40G

6x and 12x 100G

True 40G and 100G flows

Line-rate L2/L3 (IPv4 and v6)

“Fabric” features: Classic Ethernet, FabricPath, FCoE

“EARL / M-series” features: MPLS, OTV, LISP, GRE, VXLAN

FEX support

The F3 card is really the one that a lot of people have been waiting for. It brings the best of the F series together with the best of the M series into a single module based on a Switch on Chip (SoC) design. The gap between F3 and M2 at this point is very small with only these features being exclusive to M2:

Full Netflow

Big tables

7700 vs 7000

This big question is: How does this compare to the Nexus 7000?

First, a preamble. I’m not in marketing. I don’t get paid to market products and nor should I :-) I’m a network engineer and my interest is in positioning the right product for the job and designing complete solutions that address business needs. So with that in mind, I may end up repeating some of the marking talk, but that’s because I agree with how the marketing folks are positioning the switch.

The Nexus 7000 is the leader in the data center today at 10G. The marketing folks can tell you how many customers and ports and so on are out there. The numbers speak for themselves. From an “in the field” perspective, my customers are very happy with the 7000. It has great features, the software has been quite good, and the platform has been able to grow and adapt to their changing environments.

Where I see the 7700 coming into play is for customers looking to build networks at 40G and beyond. And most importantly at densities that they cannot achieve on the 7000. Other than that, the 7700 runs the same NX-OS software and has the same ASICs under the hood which really just makes it a souped-up version of the 7000. There’s nothing compelling in the rest of the 7700 if you’re a 7000 customer today and the 7000 is doing the job for you. And that’s how it’s supposed to be. It’s not a replacement of the 7000, it’s meant to fill the need for higher density 40/100G.

If the Nexus 7000 is a Cadillac CTS with a V6 engine and 304HP, the 7700 is the CTS-V with a supercharged V8 engine at 556HP. Same interior, same dashboard, same NAV/infotainment system. Just different performance levels of the same base vehicle.

Sharing Modules

The second big question: can modules be shared between the 7000 and 7700?

The “too long, didn’t read” answer: no, modules cannot be shared.

The more verbose answer involves differences in form-factor and cooling. Cooling, you say? Yes, cooling I say. The 7700 has true front-to-back airflow by way of perforations in the front of the IO modules and supervisor cards. This allows air to be drawn straight in through the front of the chassis, along the heat sinks and chips of the cards, and exhausted straight out the rear. In order for this type of cooling to be achieved, obviously the 7700 cards need to have perforations in the front of the card but they also need the components oriented so that the air can pass over the card front-to-back without impediment. By contrast, the cards for the 7000 – which is designed for side-to-side cooling – are all designed to have air pass side-to-side over the card. Heat sinks and DIMMs and so on are all oriented at 90deg relative to their 7700 counterparts.

What about the cooling on the 7010 you say? While the 7010 does offer front-to-back cooling, it’s a bit of a cheat. First, there is a large air intake duct at the bottom of the chassis which takes up about 7RU of vertical space. Being able to forego that intake means the 7710 comes in at 14RU which is the same height as the 7009 but with the advantage of an additional IO module slot and front/back cooling vs. side/side. The other aspect of the 7010 is that the cards are all turned vertical so that the air that is sucked in through the intake duct can flow side-to-side over the card (bottom-to-top of the chassis) and exhaust out the top-rear of the chassis.

To summarize: the IO modules are not interchangeable because the components on them are laid out differently in order to accommodate the different cooling architecture of each family of chassis.

As far as form-factor, the supervisor modules on the 7700 series are half-width cards. The supervisors from the 7000 simply won’t fit into a 7700 chassis.

Fabric modules? Power supplies? Fan trays? No, no, and no. A 7700 chassis needs 7700 components. Even the F2E card which is common to each chassis has a distinct part number for each chassis and cannot be interchanged between the two.

Battle Tested

I think it’s worth noting that the 7700 uses a whole lot of the same technology that underpins the 7000.

The same ASICs on the fabric modules and F2E modules

The supervisor cards have the same function and scale

They run the same branch of NX-OS software with the same feature-sets and versioning

They employ the same central arbitration and VOQ/QoS model

Of course no one wants to be the first one into the pool, but I think it’s comforting to know that the core technology of the 7700 has already been in the field for a long time. It might be a brand new model number, but the guts have already been poked, prodded and beaten up in the “real world”.

More Information

There’s no better way to get more information on the 7700 than watching Jimmy Ray dig into it :-)

9 thoughts on “An Introduction to the Nexus 7700”

How so? Many other manufacturers (Arista, Juniper, and Dell/Force10, others) offer core switching solutions with greater 10/40/100G density, performance, and certainly value. Nexus 7000 is the “leader in the data center” for Cisco-exclusive networks – by default.

The statement is perhaps worded too generically to be true on a global scale. My frame of reference is in the area of the world that I work and the customers/companies that I deal with. Within that scope, the statement holds.

This goes without saying, but the F3 is the third generation of the F-series card and the 3rd gen of that ASIC. I consider it to be quite mature.

I really like this card because, for the majority of use cases, it eliminates the decision of M vs F. It also eliminates the complexity of mixed-card VDCs. You can build a chassis now with a uniform set of cards that can do L2, L3, all the overlays, Netflow, etc. Very nice.

And if you care about these things, the Watt per Gbps on the F3 is ridiculously low compared to the M so there’s another benefit of looking at F3 for doing your L3 stuff.

OTV on the F3 does still require a separate VDC. This is true even on the latest code (7.2). I can’t comment on the roadmap here so instead I recommend you contact your Cisco SE if you want to know if/when the situation might change.

I have single Nexus 7700 with 2 x N77-F248XP-23E line cards and 2 sups. Recently we purchase 2xF3 line cards for OTV and other reasons.

My existing F2 line cards/modules have heavy production load on it. I wanted to replace these 2xF2 line cards with F3 modules and migrate all work load from F2. My all servers are multi-link between 2 line cards (port-channel)
I was thinking that I should just remove one F2 line card and replace it with F3 and copy same config on same ports. This way I will have no disruption of service. Once I am done with first F2 line card then I move the 2nd.

My question is that is it possible to have 2 modules with different types? eg. one module F2 and F3?
My current NX-OS version is 6.2? do you think this method would work? please advise.
Thanks in advance.