Data Virtualization and Sandboxes: Filling the DevOps Data Gap

Posted by admin November 30, 2016

At this year's AWS re:Invent show, the Quali Team will be showcasing our integration with Data Virtualization provider Delphix. We're super excited to show how together, Quali and Delphix fill in a huge DevOps gap: the Data Gap.

What's the Data Gap?

The Data Gap is the fact that provisioning production-like data effectively for developers and testers is one of the most challenging aspects of standing up the environments that are so critical to enabling DevOps.

Let's back up a bit to understand the context of this. DevOps is all about building, testing, and releasing software at speeds that are orders of magnitude faster than traditional methods. Enterprises used to release software (or products) on yearly or quarterly basis. Today's application based economy is forcing them to move to monthly, weekly, or daily releases. DevOps aims to transform companies' cultures, processes, and tools to enable high velocity, continuous deployments of software. In speaking about this goal, DevOps guru and Phoenix Project author Gene Kim says,

One of the most powerful things that organizations can do [to enable DevOps] is to enable developers and testers to get the environments they need when they need them.

To deliver these environments, a deployment too must to be able to automate the deployment of three key components:

Compute resources (VMs and application configurations)

Network configurations

Data

Many automation and orchestration tools focus on deploying compute resources (VMs) or configuring applications. Some focus on network configuration (though even tools like Docker have a hard time getting this right). Very few focus on data provisioning - and this is the Data Gap.

Why is the Data Gap so Important?

The Data Gap is hugely important, because data is increasingly becoming a critical component to the core IP and functionality of the software that enterprise businesses depend on internally (line of business applications) or deliver to external users (mobile banking, point of sale, online services, etc.). Furthermore, as today's digital businesses grow and expand their user base, and as applications are able to track and generate more data, the amount of data businesses acquire is growing exponentially. Because of this, businesses recognize they need to practice the DevOps principle of "continual feedback" and maximize their use of this data so they can gain competitive advantage in the marketplace and make better-informed decisions.

Quali Sandboxes and Delphix Virtual Data fill the DevOps Data Gap.

But getting developers and testers access to production or production like data is a huge challenge - fraught with manual processes, long wait times, and high risks. Enterprises can easily take days or weeks to get production data into the dev/test pipeline. And even when they do - it's fraught with risks, typified by horror stories like Patreon where customer data is leaked because development servers were hacked that were using snapshots of production data.

Filling the Data Gap: Data Virtualization and Sandboxes

Data Virtualization technologies, like Delphix, are finally solving this problem - allowing access to trusted and secured production data that available on-demand in near-real-time, with very little overhead. Data Virtualization is like an abstraction layer that allows federating many disparate data sources (structured and unstructured), managing the data with snapshots and bookmarks, and securing and cleansing the data.

Now, if you can get that virtualized data in to sandboxes, then you can close the Data Gap. Why? Because sandboxes deploy all three key components of DevOps environments: compute, network, and data. Quali's cloud sandboxes give developers and testers on-demand access to replicas of production environments. They allow you to provision full-stack environments that include all the critical components that developers and testers need. This goes beyond just the application under test (AUT) or device under test (DUT). Sandboxes deploy virtual resources, cloud components, physical equipment/devices, networking, and data. Sandboxes also provision ancillary resources like test tools, service virtualization components, and third party interfaces.

Stop by booth #2832 to see how Quali sandboxes and Delphix data virtualization fill the DevOps Data Gap and enable you to deliver full-stack production environments to dev and test in minutes rather than days or weeks.