Random thoughts and technical bits

Pernix Data 30 days later

I have been interested in Pernix data since its initial release the idea of using flash to accelerate storage is not new to me. Anyone who reads my blog based rants have found that I am a huge supporter of larger cache on storage arrays. I have always found having more cache will make up for any speed issues on drives. My thoughts on this are simple if 90% of my storage writes and reads come from cache I run at near the speed of the cache. Spend your money on larger cache instead of faster spinning disks and performance is improved. Almost every storage array vendor has been using ssd to speed up arrays for the last four years. They all suffer from the same problem, they treat all I/O’s equal without any knowledge of workload.

Netfliks problem

The best way to explain this problem is using netflicks. They have implemented a system where you rate a show based upon stars. Then it compares your ratings against everyone else ratings and locates users with similar ratings to yours. Once located it uses these user’s recommendations to locate new shows for you. This is great… assuming you have 100% the same taste in shows as that user. This has been advanced a lot in the last five years and is much more accurate and complex algorithm. It’s pretty accurate for me except for one problem… My wife and kids share the same netfliks account and my children love to rate everything. This produces the world worst set of recommendations… I get little girl TV shows mixed with Downton Abbey and Sci-Fi movies. It’s a mess… Netflix literally has no idea how to recommend show to me. This problem exists for storage arrays with cache. Choosing which data should be cached for reads is hard, because there are lots of different workloads competing for cache. I don’t want to devalue the algorithms used by storage vendors, they much like Netflix are a work of evolving art. With everyone being profiled into one mass everyone’s performance suffers. Netflix understood this problem and created user profiles to solve the problem. They added simple versions of localized intelligence to the process. These pockets of intelligent ratings are used to provide recommendations for the local needs.

Pernix is the intelligent user profile

Pernix is just like Netflix user profiles, it’s installed locally on each ESXi server. It caches for that ESXi host (and replicates writes for others). It can be configured to cache everything on the host, datastore or virtual machine. It provides the following features:

The only local SSD write cache that I know of outside hyper-converged solutions

Local SSD read cache

Great management interface

Metrics on usage

Replication of writes to multiple SSD’s for data protection

Pernix is built for vSphere

Pernix installs as a VIB into the kernel and does not require a reboot. It has a Web client interface and C# client interface. It does require a Windows server and SQL server for reporting. It is quick and easy to install and operate. The cache can be SSD’s or memory for pure speed. Pernix works only in vSphere so it’s 100% customized for vSphere.

Lab

My local Pernix SE’s were kind enough to provide me a download and license for Pernix Data. My home lab has been documented on this blog before but the current solution is 3 hp nodes with 32Gb of RAM each as shown below:

I added a 120GB san disk SSD to each node for this test. My storage ‘array’ is an older Synology nas with two mirrored 2TB 7,200 RPM disks via iSCSI and NFS. My rough math says I should be getting about 80 IOPS total from this solution which really sucks, oddly it’s always worked for me. I didn’t have any desire to create artificial workloads for my tests, I just wanted to see how it accelerated my every day workload. All of these tests were done in vSphere 5.5 U2.

Pernix Look and feel

Pernix provides a simple and powerful user interface. I really like the experience even in the web client. They use pictures to quickly show you where problems exist.

As you can see lots of data is presented in a great graphical interface. They also provide performance charts on every resource using Pernix. Without reading any manual other than Pernix quick start guide I was able to install their solution in 15 minutes and have it caching my whole environment, it was awesome.

How do we determine storage performance?

This is a constant question, every vendor has a different metric they want to use to determine why their solution is better. If it’s a fiber channel array they want to talk about latency then IOPS. If it’s all flash NAS its IOPS then latency. So we will use these two metrics for the tests:

Latency – time it takes to commit a write or get a read

IOPS – Input / Outputs per second

I wanted to avoid using Pernix’s awesome graphs for my tests so I chose to use vRealize Operations to provide all recorded metrics.

Baseline

The VM that gives my environment the biggest storage workout is vRealize log insight. It has been known to have recorded IOP’s of 300 in the environment. Generating IOP’s is easy just click around the interface prebuild dashboards with the time slider set for all time. Read IOP’s fly up like crazy. So my average information before Pernix is as follows:

Max IOPS 350

Max Latency: 19 ms

Average Latency: 4 ms

Now with Pernix

I setup Pernix to cache all virtual machines in my datacenter. With pernix I clicked around on multiple days and performed lots of random searches. I loaded down a SQL server with lots of garbage inserts to create writes. Nothing perfectly scientific with control groups I just wanted to kick the tires. After a month with pernix I got the following metrics:

Max IOPS: 4,000

Max Latency: 14 ms

Average Latency: 1.2 ms

So the results clearly denote a massive increase in IOP’s. Some may say sure you are using SSD’s for the first time, which is true. The increase is not just SSD’s speed because the latency is greatly improved as well which is representative of the local cache. Imagine using enterprise worthy SSD’s with much larger capacity. Simple answer will Pernix improve storage performance… the answer is it depends but there is a very good chance.

Use Cases

With my home lab hat removed I need to talk about some enterprise use cases:

Any environment or workload where you need to reduce latency

Any environment where workload needs every more IOP’s than most of the solutions

Both of these use cases should be implemented where less latency or IOP’s is a direct cost. Pernix can be used as a general speed enhancer on some slower environments or to improve legacy arrays. It does push toward a scale up approach to clustering. Larger cluster nodes with larger SSD’s will cost less than lots of nodes. Pernix is licensed per node. Putting in larger nodes does have a big impact on failure domains that should be taken to account.

My only Gripe

My only gripe with Pernix is the cost. Compared to large storage array’s it is really cheap. The problem is budgets… I need more storage performance which means the storage team buys more storage arrays not the compute team. Getting that budget transferred is hard because storage budgets are thin already. This will change hyper-converged is becoming very accepted and Pernix will really shine in this world. Pernix just released the read cache for free making it a very tempting product. They are a smart company with a great product. They are on the right path bringing storage performance as close to the workload as possible with an added element of intelligence.

About Author

Joseph Griffiths is a virtualization focused solutions architect who works with complex cloud based solutions. He currently holds many IT certifications including VMware VCDX-DCV and VCDX-CMA #143. This blog represents his random technical notes and thoughts. The thoughts expressed here do not reflect Joseph’s current employer in anyway. You can follow Joseph on Twitter @Gortees