It’s that time of year again, time for NetApp’s annual technical conference, Insight. This also means that a Long-Term Support (LTS) release of ONTAP is due, this time it’s 9.5. As I write this, I am sitting in the boarding lounge of YVR, waiting for my flight to Las Vegas for NetApp Insight and I see the Release Candidate (RC) for 9.5 is not out quite yet, but I do have the list of new features for you nonetheless.

The primary new features of 9.5 are:

New FlexCache accelerates performance for key workloads with read caching across a cluster and at remote sites.

FlexCache: FlexCache makes its return in 9.5 and provides the ability to cache hot blocks, user data and meta data on a more performant tier while the bulk of the data sits in a volume elsewhere in the cluster or even on a remote cluster. FlexCache can enable you to provide lower read latency while not having to store the bulk of your data on the same tier. At this time, only NFSv3 is supported though the source volume can be on AFF, FAS or ONTAP Select. While the volume you access is a FlexGroup volume, the source volume itself cannot be a FlexGroup but rather must be a FlexVol. An additional license is required.

SnapMirror Synchronous: SM-S also makes a long-awaited return to ONTAP allowing you to provide a recovery point objective (RPO) of zero and very low recovery time objective (RTO). FC, iSCSI and NFSv3 only at this time and your network must have a maximum roundtrip latency of no more than 10ms, FlexGroup volumes not supported. An additional license is required.

MetroCluster-IP (MC-IP): NetApp continues to add value to the mid-range of appliances by bringing MC-IP support to both the AFF A300 as well as the FAS8200. At the same time, NetApp has increased the maximum distance to 700km, provided your application can tolerate up to 10ms of write acknowledgement latency.

FabricPool: Previously hampered by the need to tier volumes greater than 100TiB? Now that FabricPool supports FlexGroups, you are in luck. Also supported in 9.5 is end-to-end encryption of data stored in FabricPool volumes using only one encryption key. Lastly, up until now, data would only migrate to your capacity tier once your FabricPool aggregate reached a fullness of 50%, this parameter is now adjustable though 50% remains the default.

While those are the primary features included in this latest payload, existing features continue to gain refinement, especially in the realm of storage efficiency. Specifically, around logical space consumption reporting, useful for service providers. Also, adaptive compression is now applied when 8KB compression groups (CG) are <50% compressible, allowing CG’s to be compacted together. Databases will see the most benefit here, typical aggregate savings in the 10-15% range. Finally, provided you have provisioned your storage using System Manager’s application provisioning, adaptive compression will be optimized for the database being deployed; Oracle, SQL Server or MongoDB.

That’s all for now, if you want more details come find me at NetApp Insight on the show floor near the Social Media Hub or at my Birds of a Feather session, Monday at 11:15am where myself and other NetApp A-Team members will discuss the Next Generation Data Centre.

As NetApp continues to make its mark on and help define the Next Generation Data Centre, the need for more node types of their HCI offering has become apparent and they are responding in kind.

First up, staying current by using the latest generation of Intel Skylake processors in the new nodes is a given; as well as offering myriad combinations of both CPU and memory while maintaining interoperability with the current generation of HCI nodes.

First up, are a raft of new compute nodes, some of which are optimized around core count which you can use to satisfy various licensing models.

Model #

Processor

Memory

H410C-14020

2 x Xeon Silver 4110
(8 core @ 2.1GHz)

384 GB

H410C-15020

512 GB

H410C-17020

768 GB

H410C-25020

2 x Xeon Gold 5120
(14 core @ 2.2GHz)

512 GB

H410C-27020

768 GB

H410C-28020

1 TB

H410C-35020

2 x Xeon Gold 5122
(4 core @ 3.6GHz)

512 GB

H410C-37020

768 GB

H410C-57020

2 x Xeon Gold 6138
(20 core @ 2.0GHz)

768 GB

H410C-58020

1 TB

Next up, the much-requested GPU accelerated compute nodes have been announced, optimized for Windows 10 VDI deployments. This one moves away from the 2 RU chassis with 4 compute nodes and is one 2 RU server in itself consisting of:

2 x NVIDIA Tesla M10 GPUs

An Intel Skylake Xeon 6130 (16 cores @ 2.1GHz)

512MB RAM

On to the networking-side of things, your concerns have been heard. NetApp will soon begin offering their H-Series switch, the Mellanox SN2010 to help complete your HCI build-outs. This switch is a paltry 1RU, half-width consisting of 18 SFP+/28 ports with optional cable and transceiver bundles. Support for this switch will be NetApp-direct, so no worries around cross-vendor finger pointing.

Keeping in the network mindset, NetApp is making things simpler by reducing the required network port count and associated infrastructure by 40%. HCI compute nodes now only require two SFP28 connections, down from four, vSphere distributed switch is a requirement.

While NetApp keeps improving the front end that is ActiveIQ, for both pre-sales and support purposes, I constantly find myself going into the Classic AutoSupport and accessing the raw autosupport data; most often it’s sysconfig -a. Recently I was trying to explain the contents to a co-worker and I realized that I should just document it as a blog post. So here is sysconfig explained.

The command sysconfig -a is the old 7-mode command to give you all the hardware information from the point of view of ONTAP. All the onboard ports are assigned to “slot 0” whereas slot 1-X are the physical PCIe slots where myriad cards can be inserted. Here’s one example, I’ll insert comments as I feel it is appropriate. Continue reading →

While the actual payload hasn’t hit the street yet, here’s what I can tell you about the latest release in the ONTAP 9 family which should be available here any day now. **EDIT: RC1 is here.9.4 went GA today.

FabricPool

Lots of improvements to ONTAP’s object-tiering code in this release, it appears they’re really pushing development here:

Support for Azure Blob, both hot and cool tiers, no archive tier support

This adds to the already supported AWS-S3 and StorageGRID Webscale object stores

Support for cold-data tiering policies, whereas in 9.2,9.3 it was backup and snapshot-only tiering policies

Default definition of cold data is 31 days but can be adjust to anywhere from 2-63 days.

Not all cold blocks need to be made hot again, such as snapshot-only blocks. Random reads will be considered application access, declared hot and written back into performance tier whereas sequential reads are assumed to be indexers, virus scanners or other and should be kept cold and therefore will not be written back into performance tier.

Now supported in ONTAP Select, in addition to the existing ONTAP and ONTAP Cloud. Wherever you run ONTAP, you can now run FabricPools, SSD aggregate caveat still exists.

Inactive Data Reporting by OnCommand System Manager to determine how much data would be tiered if FabricPools were implemented.

This one will be key to clients thinking about adopting FabricPools

Object Store Profiler is a new tool in ONTAP that will test the performance of the object store you’re thinking of attaching so you don’t have to dive in without knowing what your expected performance should be.

Object Defragmentation now optimizes your capacity tier by reclaiming space that is no longer referenced by the performance tier

Compaction comes to FabricPools ensuring that your write stripes are full as well as applying Compression, Deduplication

It’s that time of year again, time for an ONTAP release…or at least an announcement. When 9.3 drops, not only will it be an LTS (Long Term Support) version, but NetApp continues to refine and enhance ONTAP.

Simplifying operations:

Application-Aware, data management for MongoDB

Adaptive QoS

Guide cluster setup and expansion

simplified data protection setup, much simpler.

Efficiencies:

Not so long ago, in ONTAP 9.2, NetApp introduced inline, aggregate-level dedupe. What many people may not have realized, due to the nature of way ONTAP coalesces writes in NVRAM prior to flushing them to the durable layer is that this inline aggregate dedupe’s domain was restricted to the data in the NVRAM. With 9.3, a post-process aggregate scanner has been implemented to provide true, aggregate-level dedupe.

Continuing with their new standard six month release cadence, ONTAP 9.2RC1 was released today and I continue to be impressed with the feature payload NetApp has been delivering with each new release; here are the highlights:

FabricPools

Automated cloud-tiering of NetApp Snapshots to a target that speaks S3 (AWS or StorageGrid)

QoS Floors or minimums

Allows you to reserve performance for critical workloads, SAN on AFF only.

­With the introduction of the new NetApp platforms back in September 2016, came 40GbE as well as 32Gb Fibre Channel connectivity.

I had my first taste of 40GbE on the NetApp side back in January when I got to install the first All Flash FAS A700 in Canada. The client requested a mix of 40GbE and 16Gb FC with some of the 40GbE being broken out into 4 × 10GbE interfaces and some being used natively.

NetApp is deploying two flavours of 40GbE cards: the X1144A for the AFF A300, AFF A700s and FAS8200, and the X91440A for the AFF A700 and FAS9000 storage systems. At first glance, you might be tempted to assume that those are the same PCIe card since the part numbers are very similar (the latter just being in some sort of carrier to satisfy the I/O module requirement for the blade-style chassis that is home to the A700 and FAS9000), Upon further inspection the two are not exactly equal.

The ports on most PCIe cards and onboard interfaces are deployed in pairs, with one shared application-specific integrated circuit (ASIC) on the board behind the physical ports. On the X1144A, both external ports share one ASIC with an available combined bandwidth of 40Gb/s, whereas the X91440A has two ASICs. Each has two ports, but one is internal and not connected to anything, giving you 40Gb/s per external port.

The ASIC (or controller) in question is the Intel XL710. What’s important about this is that both external ports on an X91440A can be broken out to 4 × 10GbE interfaces for a total of eight, or one can remain at 40GbE while the other is broken out. On the X1144A however, you can either connect both ports to your switch using 40GbE connections or you can break-out port A to 4 × 10GbE and port B gets disabled. According to Intel, if you connect both ports via 40GbE, “The total throughput supported by the 710 series is 40 Gb/s, even when connected via two 40 Gb/s connections.”

Now before we get all up in arms about this, lets really get into the weeds here. Both the FAS8200/FAS9000 and the AFF A300/700 are using PCIe 3.0. Each PCIe 3.0 lane can carry 8 Gigatransfers per second (GT/s). For the purposes of this post, that is close enough to 8Gb/s. The FAS8200/AFF A300 has an Intel D-1587 CPU with a maximum eight lanes per slot, so roughly 64Gb/s of throughput, whereas the FAS9000/AFF A700 has an Intel E5-2697 with a maximum 16 lanes per I/O slot which gives it about 128Gb/s of throughput. So even if NetApp included a network interface card for the A300/FAS8200 with two XL710’s on it, the PCIe slot it’s connected to couldn’t provide 80Gb/s of throughput, whereas the the I/O modules in the A700/FAS9000 can.

Say you want to change between 40GbE and 10GbE. Unlike modifying UTA2 profiles (as explained here), with the XL710, you need to get into maintenance mode first and use the nicadmin command. Here’s an example:

Unfortunately I don’t have access to either a FAS8200 nor an AFF A300 with 40GbE otherwise I’d provide the sysconfig output before and after there as well.

Now, there’s a bit of a debate going on around the viability of 40GbE over 100GbE. While 40GbE is simply a combined 4 × 10GbE; 100GbE is only a combined 4 × 25GbE. With regards to production costs, apparently to make a 40GbE QSFP+, you literally combine 4 lasers (hence the Q in QSFP) into the module; well, the same goes for 100GbE. You only need one laser to produce the wavelength for 25GbE, and while that still means you need four for 100GbE, four times the production cost still yields 250% of the throughput of 40GbE which makes me wonder where it will end up in a year.

So there you go, more than you ever wanted to know about NetApp’s recent addition of 40GbE into the ONTAP line of products as well as my personal philosophical waxing around the 40 versus 100 GbE debate.

It all begins in the configuration builder tool

This article focuses on the implementation and management of encryption with NetApp storage. Data at Rest Encryption (NetApp Volume Encryption or NVE for short) is one of the ways that you can achieve encryption with NetApp, and it’s one of the most exciting new features of ONTAP 9.1. Here’s how you go about implementing it.

If you’re a partner or NetApp SE, when building configurations, as long as the cluster software version is set to 9.x, there is a checkbox that lets you decide which version of ONTAP gets written to the device at the factory. As of 9.1, ONTAP software images will either be capable of encryption via a software encryption module, or not. There are laws around both the import and export of software that is capable of encryption, but that is beyond the scope of this article. I do know you can use the encryption-capable image in Canada (where I am located), so I’m covered. If you’re unsure about the laws in your country, consult your legal adviser on this matter.

Once this cluster-level toggle has been set and you add hardware into the configuration, there are two more checkboxes in the software section:

The first one triggers the generation of the license key for NVE and the second one activates a piece of hardware dedicated to deal with cryptographic keys. One thing I’m still not sure of is (should you choose to remove the checkmark) if the TPM is simply disabled or doesn’t physically exist in your NetApp controller, I have an email into NetApp to confirm this. [Update: The module is integral to the controller and disabled in firmware if being shipped to certain countries. Shout out to @Keith_Aasen for tracking that down for me.]

Okay, now for the more customer-relevant information…

To get started with NVE, you’re going to need a few things:

A encryption-capable platform

A encryption-capable image of ONTAP

A key manager

A license key for NVE

Encryption-capable platform

The following platforms are currently capable of encryption: FAS6290, FAS80xx, FAS8200, and AFF A300. This is limited by the CPU in the platform as it must have a sufficient clock-speed and core-count with support for the AES instruction set. I’m sure this list will be ever-expanding, but be sure to check first if you’re hoping to use NVE. [UPDATE: After some digging, I can confirm that all the new models support NVE, the entry-level FAS2650 included.]

Encryption-capable image of ONTAP

Provided you’re not in a restricted country as per the above, your image will be the standard nomenclature of X_q_image.tgz where X is the version number. The non-encryption-capable version will be X_q_nodar_image.tgz which I’ll simply refer to as nodar(e) (No Data At Rest Encryption) for the rest of this article. The output of version -v will tell you if you’re nodar or standard.

Key manager

The on-board key manager introduced in ONTAP 9.0 enables you to manage keys for use with your NSE drives, helping you avoid costly and possibly complex external solutions. Currently, NVE only supports using the on-board manager, so if you’re going to use NVE layered on top of NSE, you need to use the on-board one.

Setting this up is exactly one command:

security key-manager setup

You’ll be prompted for a passphrase, and that’s it, you’re done.

License key for NVE

If you didn’t get this license key at time of purchase, talk to your account representative or SE over at NetApp (though, hopefully, if you’ve bought one of the new systems announced at Insight 2016, they decided to include it since, at least for now, it is a no-cost license).

What next?

Now that you’ve got all the prerequisites covered, encrypting your data is very simple. As the name implies, encryption is done at the volume level, so naturally it’s a volume command that encrypts the data (a volume move command, in fact):

Wrapping up

NetApp Volume Encryption is pretty easy, but since it’s so new, OnCommand System Manager doesn’t support it just yet. You’ll have to stick to the CLI for now, although I’m sure the GUI will catch up eventually, if that’s your preferred point of administration. It should also be noted that while NSE solutions are FIPS 140-2 compliant, NVE has yet to go through the qualifications. Also, if FIPS is a requirement, the on-board key manager isn’t compliant yet either. Since with the on-board key manager the keys are literally stored on the same hardware using them, NVE only protects you from compromised data on individual drives removed from your environment through theft or RMA. If someone gained wholesale access to the HA pairs, the data would still be retrievable. Also, this is for data-at-rest only. You must follow other precautions for data-in-flight encryption.

Into the weeds

I did all my tests for this post using the simulator, and I learned a lot, but your mileage may vary. In the end, only you are responsible for what you do to your data. I had heard that if you have the wrong software image then you’d have to do a complete wipe of your HA pair in order to convert it. I have since proven this wrong (at least in the simulator) and I definitely can’t guarantee the following will be supported.

For my tests I had two boot images loaded: one standard and one nodar. What I learned is that you can boot into either mode, provided you don’t have any encrypted data. Even if you have the key manager setup and NVE is licensed, you can still boot back and forth. The first time you boot your system using the nodar image with encrypted data on the system, however, you’ll hose the whole thing. I did test first encrypting data, then decrypting it, then converting to nodar, and the simulator booted fine. When I booted into nodar with an encrypted volume, even going back to standard didn’t work. Booting into maintenance mode shows the aggregates with a status of partial and the boot process hints that they are in some sort of transition phase (7MTT?). Either way, I was unable to recover my simulator once I got it to this state, so I definitely advise against it in production. Heck, I’d advise you just to use the proper image to start with.

I hope you learned something. If you have any questions or comments, either post them below or reach out on twitter. I’m @ChrisMaki from the #NetAppATeam and Solution Architect @ScalarDecisions.

Since Insight Berlin is now done and several of my fellow A-Team’ers have already done their own “Insight wrap-up” blogs, and you can check them out for a detailed run down of all the awesome stuff that NetApp is coming out with in the next year, I figured I should get mine done as well (posts from @mcbride_ruairi and @NFSDudeAbides to start you off). While all that stuff is really cool (ONTAP 9, new All Flash FAS, etc.), what was the most inspiring for me from this year’s conference is how NetApp is bringing the “vision” of the Data Fabric to life with a couple of key advancements.

Flash (and ONTAP) Everywhere

Since 2014, when George Kurian announced the Data Fabric on stage at Insight, NetApp has really pushed its flash portfolio in new directions. The subsequent acquisition of SolidFire has bolstered NetApp’s position in the market, while giving customers more options and improved capabilities for deploying flash in their data centers. This year, George set the goal (albeit a bit lofty) of being the number one flash vendor in the world, and with that attitude and the right strategy, they seem like they’re on their way.

Of course, the big to-do around flash this year at Insight was the refresh of the All Flash FAS line of storage systems. It was the biggest hardware refresh in NetApp’s history, with the introduction of the new FAS2600 and FAS8200 lines as well as the new AFF A300 and AFF A700 systems. Along with this comes 32Gb Fibre Channel and 40Gb Ethernet, which is a first for the SAN world and helps position the storage better for flash.

But instead of these announcements just being the shiny new toys for engineers to ooh and ah over, NetApp brought them to market with the strategic intent to help customers access their data anywhere. The importance of data was a central element throughout the conference, with the “data is the currency of the digital economy” message resonating loud and clear. One of my favorite quotes from the event was, “[In the past,] data was just there to run your business. Now data is your business.”

With the increased capabilities and applicability of flash comes the more pronounced ubiquity of ONTAP in the Data Fabric. NetApp’s strategy has always been to help customers move and utilize data where it can deliver the most value to them, and this year they are making some big strides in that direction. Tools like Flash Cache, Flash Pool, SnapMirror, SnapVault, and others are being expanded across the portfolio to deliver a seamless data management experience.

Another cool piece of software that they’re bringing out is called Cloud Control, which enables you to back up your data that resides with SaaS providers within SnapCenter, things like Office 365, with support for other SaaS providers to come. They showcased a pretty fancy-looking SnapCenter with cataloguing and all sorts of other cool features to help you manage cloud data like you would your on-premises storage. SnapCenter is currently free, but with how cool Cloud Control is gonna be, I honestly have no idea why they wouldn’t want to monetize it.

Renewed Focus on Developers, a Sign of Greatness to Come

This idea of an open, flexible, and secure ecosystem for your data is only going to get more crystalized as time goes on. Something that my fellow A-Teamer Jesse Anderson blogged about a while back that struck me as well was NetApp’s focus on the developer at Insight. I think it’s a sign of a cultural shift that will only further enable the Data Fabric. The introduction of thePub, this place where developers can coalesce and share ideas to make the Data Fabric better, is a huge step in the right direction as far as I’m concerned.

I’m really excited to see what that future has in store for NetApp and the Data Fabric. See you next year!

Ever since clustered Data ONTAP went mainstream over 7-Mode, the dedicated root aggregate tax has been a bone of contention for many, especially for those entry-level systems with internal drives. Can you imagine buying a brand new FAS2220 or FAS2520 and being told that not only are you going to lose two drives as spares, but also another six to your root aggregates? This eﬀectively left you with four drives for your data aggregate, two of which would be devoted to parity. I don’t think so. Now, this is a bit of an extreme example that was seldom deployed. Hopefully you had a deployment engineer who cared about the end result and would use RAID-4 for the root aggregates and maybe not even assign a spare to one controller, giving you seven whole disks for your active-passive deployment. Still, this was kind of a shaft. In a 24-disk system deployed active-active, you’d likely get something like this:

Enter ADP.

In the ﬁrst version of ADP introduced in version 8.3, clustered Data ONTAP gained the ability to partition drives on systems with internal drives as well as the ﬁrst two shelves of drives on All Flash FAS systems. What this meant was the dedicated root aggregate tax got a little less painful. In this ﬁrst version of ADP, clustered Data ONTAP carved each disk into two partitions: a small one for the root aggregates and a larger one for the data aggregate(s). This was referred to as root-data or R-D partitioning. The smaller partition’s size depended on how many drives existed. You could technically buy a system with fewer than 12 drives, but the ADP R-D minimum was eight drives. By default, both partitions on a disk were owned by the same controller, splitting overall disk ownership in half.

You could change this with some advanced command-line trickery to still build active-passive systems and gain two more drive partitions’ worth of data. Since you were likely only building one large aggregate on your system, you could also accomplish this in System Setup if you told it to create one large pool. This satisﬁed the masses for a while, but then those crafty engineers over at NetApp came up with something better.

Enter ADPv2.

Starting with ONTAP 9, not only did ONTAP get a name change (7-Mode hasn’t been an option since version 8.2.3), but it also gained ADPv2 which carves the aforementioned data partition in half, or R-D2 (Root-Data,Data) sharing for SSDs. Take note of the aforementioned SSDs there, as spinning disks aren’t eligible for this secondary partitioning. In this new version, you get one drive back that you would have allocated to be a spare, and you also get two of the parity drives back, lessening the pain of the RAID tax. With a minimum requirement of eight drives and a maximum of 48, here are the three main scenarios for this type of partitioning.

12 Drives:

24 Drives:

48 Drives:

As you can see, this is a far more eﬃcient way of allocating your storage that yields up to ~17% more usable space on your precious SSDs.

So that’s ADP and ADPv2 in a nutshell—a change for the better. Interestingly enough, the ability to partition disks has lead to a radical change in the FlashPool world called “Storage Pools,” but that’s a topic for another day.