The SDS Landscape

Scott D. Lowe - ActualTech Media

This post is part of a series based on the forthcoming book, ​Building a Modern Data Center​, written by Scott D. Lowe, David M. Davis and James Green of ActualTech Media in partnership with Atlantis Computing.

What does and doesn’t qualify as software-defined storage is a topic of discussion amongst many IT professionals. Fortunately, the answers are pretty straight forward. In this article, we’ll look at what is and isn’t really software-defined, and also look at the two primary ways that SDS is enabled from a technology standpoint.

What Is Software-Defined Storage?

The exact definition of software-defined storage is still evolving, but the generally accepted definition is that SDS is where the management and intelligence of the storage system is decoupled from the underlying hardware storage system. The intelligent software is then able to provide policy-based management and provisioning of the data being stored regardless of the storage hardware that’s being used.

Today’s SDS systems include storage virtualization that provides this abstraction of the storage intelligence away from the underlying storage. SDS systems typically allow consumers great flexibility in the underlying storage hardware and advanced storage features. For example, they may provide deduplication, compression, thin provisioning, snapshots, replication, caching, and other advanced storage functionality.

Figure 1 - Software Defined Storage Layers (Atlantis USX Example)

SDS may be sold separately from the underlying storage, or it may be included with the underlying storage. The most important thing is that the software that makes the storage possible can be separated from the storage hardware. In many cases, SDS runs on top of different operating systems and is compatible with multiple types of underlying storage hardware.

Along with software-defined networking, software-defined storage is a critical piece of the software-defined data center (SDDC) ideal that most companies are striving to adopt.

What Isn’t Software-Defined Storage?

Traditional storage area networks (SAN) and network attached storage (NAS) systems that are packaged as a single solution where the storage intelligence and the hardware are coupled so tightly that neither of them can be interchanged, are not SDS. Additionally, to be SDS, you must be able to both manage storage and be able to create usable storage.

SDS Compared to Traditional Storage

If software-defined storage provides so many benefits and such tremendous flexibility, why haven’t storage solutions always been architected and delivered in this way? The reason that these so-called “hardware-defined storage systems” were created in the first place was because, at the time, the server hardware that could have run this storage intelligence simply wasn’t adequate to provide the processing and throughput that the applications required of it. The rest - dedicated SAN and NAS hardware - was created in order to tightly couple the software with specialized hardware to provide high-performance.

Today with server CPU, bus, and I/O channels being able to offer such high performance, there is no reason that intelligent software can’t run on just about any x86 server in the modern data center and provide the storage services that are required by the applications.

If you were to compare SDS to traditional SAN/NAS solutions, you would find the following differences:

Hardware Flexibility — SDS runs on existing servers or on commodity servers, which are available from many sources and at more affordable costs.

Simplified Administration — SDS solutions are typically administered from a simplified web-based interface that can be understood by most IT professionals in short order.

Simplified Configuration — SDS typically sees storage as a single pool instead of managing storage through the construct of logical unit numbers (LUNs).

Advanced Features Included — SDS typically includes advanced features in the product (such as deduplication, compression, replication, and others).

New Features Included — Just like a smart phone, SDS typically includes any new features and functionality whenever a new software patch or update is released.

Greater Integration — Because SDS is software running on commodity hardware just like other operating systems and applications, it stands a greater chance of being able to be integrated into existing data center systems. For example, SDS can be easily integrated with the virtualization hypervisor and virtualization management system through the use of APIs (or application programmable interfaces).

Lower Overall Costs — In most cases, due to the elimination of dedicated, complex, and costly hardware, SDS will almost always be lower cost, overall. This is true because with SDS you can use commodity hardware since you aren’t tied into a costly service contract, and there is lower complexity for administration and troubleshooting.

Software-defined storage solutions can be deployed in a few different ways. Just like other software, SDS must run on top of some type of operating system and file system. Thus, SDS can be deployed two different ways: hypervisor/kernel-integrated or via a virtual storage appliance (VSA).

Hypervisor/Kernel-Integrated Storage

With hypervisor/kernel-integrated storage, the SDS solution runs inside either the kernel of a virtualization hypervisor or inside the kernel of another operating system.

The benefits of this type of SDS deployment are that the storage software has direct access to the operating system and thus, the underlying storage hardware. Some have claimed that hypervisor/kernel-integrated storage offers a slight performance benefit over the VSA approach (discussed more below) however the real performance difference in SDS are determined by the physical storage backing the software-defined storage intelligence, and the type/amount of caching.

The drawback to using this type of SDS deployment is that the SDS solution is typically only compatible with the kernel of the virtualization hypervisor or operating system that it’s integrated with. What that means to enterprises is that hypervisor/kernel-integrated storage solutions are version-locked, and in some cases feature-constrained, due to this level of deep integration. Additionally, some hypervisor/kernel-integrated storage solutions only support local storage, which limits your storage options.

Another drawback is that these types of SDS deployments may require the purchase of specific and costly virtualization hypervisor licenses that are required to run the SDS solution, and that don’t offer immediate value to the enterprise. All of these drawbacks are typically summed up into the single term, “hypervisor lock-in.”

VMware’s Virtual SAN (VSAN) is one example of a hypervisor/kernel-integrated SDS solution. VSAN is integrated into the vSphere hypervisor, is incompatible with other hypervisors, and requires that you license vSphere for each of your hosts in the SDS cluster. Another example of a hypervisor/kernel-integrated storage solution is Microsoft Storage Spaces.

Virtual Storage Appliances (VSA)

For software-defined storage solutions that utilize a VSA, the SDS solution runs inside the kernel of an operating system which, in turn, runs inside of a virtual machine. This virtual machine, with its operating system and SDS running inside, is what is termed a virtual storage appliance, or VSA.

While a single VSA/VM could be termed software-defined storage, in order to truly provide a modern enterprise-grade SDS solution using a VSA, you would typically run a minimum of 2 or 3 VSA/VMs across 2 or 3 different hosts for data redundancy and high availability (just as would be required for high availability in hypervisor/kernel-integrated storage). Different VSA solutions have different requirements for the minimum number of nodes necessary to achieve high availability. Some can operate with as few as two nodes and, as such, are the most economic. Others require three nodes.

The benefits of the VSA approach include:

Compatibility — with multiple virtualization hypervisors or operating systems, giving the enterprise great flexibility on where they run the SDS and at what cost

Ability to upgrade and/or change the underlying virtualization or hypervisor — without affecting the storage system. Of course, the VSA vendor must have the ability to support the hypervisor choice you may make.

Flexibility — to use lowest cost virtualization hypervisor or operating systems to run the SDS on top of, as it makes sense for the business.

With the VSA approach, the hypervisor is essentially commoditized, allowing the business to choose whatever solution is the lowest cost or makes the most sense. No matter which SDS design you leverage (hypervisor/kernel-integrated software-defined, or VSA) the high-level benefits of SDS still apply, and you’d be far better off than you would be leveraging traditional storage solutions.

SDS and Hyperconvergence

SDS is one of the major components in the hyperconverged infrastructure architecture. In the next article in this series, we’ll take a look at how SDS enables hyperconvergence, and explore some different options regarding hyperconvergence that a potential consumer needs to understand. Much the same as SDS has options like in-kernel versus VSA-based storage platforms, hyperconvergence has models like appliance-based versus software-based to consider. We’ll cover those decisions, and more.

SCOTT D. LOWE – Contributor​

​Scott is Co-Founder of ActualTech Media and serves as Senior Content Editor and Strategist. Scott is an enterprise IT veteran with close to twenty years experience in senior and CIO roles across multiple large organizations. A 2015 VMware vExpert Award winner, Scott is also a micro-analyst for Wikibon and an InformationWeek Analytics contributor.