Conflict and Irrelevance in NoSQL Land

The NoSQL market is red hot, and some say the time to capture it is right now.

While some vendors in the space think claiming superiority over the other NoSQL guy is crucial to winning markets and minds, others focus on displacing Oracle, IBM and Microsoft.

Others want to befriend the giants (or hitch a ride on their coattails).

Still others want to shine the spotlight on what they do best and let the rest take care of itself.

Have a headache yet? We’re pairing it down just a bit.

Who Takes the High Road?

Time will tell if trashing your competitors (or just pointing out their shortcomings) is the best approach.

Popular open source NoSQL database maker MongoDB used to turn the other cheek when it came to taking blows from would-be rivals. But now silence no longer seems golden.

“Couchbase’s public statements make MongoDB look broken,” said Kelly Stirman, VP of strategy and product marketing at MongoDB told CMSWire in an interview last week. “It’s as if they’re comparing a pickup truck to a sports car.”

When a competitor talks like that about you, there may be only one thing to do — let an independent party establish the facts. So it goes to follow that MongoDB commissioned United Software Associates, an independent benchmarking and performance testing organization, to compare what some consider to be the top three NoSQL vendors: MongoDB, Couchbase and DataStax.

It doesn’t take a genius to surmise that MongoDB won. (This is not to suggest that we question the outcome, but why else would they share the results with the press?)

Couchbase didn’t take long to respond, as Stirman predicted. “Their marketing messaging seems to revolve around being better than MongoDB,” he said. “Take that away, and then what?”

Couchbase, DataStax Strike Back

For a benchmark to be valid “it must be transparent and repeatable,” Shane Johnson, senior product marketing manager at Couchbase commented in response to our article, citing MongoDB’s performance statistics.

In a blog post, he pointed out shortcomings in the approach United Software Associates used to compare the three vendors. He also referred us to a benchmark test that Couchbase had done on its behalf. You can guess who won that one.

Earlier this year, DataStax commissioned EndPoint, an expert database and ecommerce consulting company, to perform a benchmark study between Cassandra (DataStax is the commercial vendor that supports it), MongoDB, Couchbase and HBase.

You know who fared best.

But to DataStax’s credit, it not only highlighted the specific area where Cassandra won the race (throughput and latency), but it also asked EndPoint to rerun the test when configuration errors were discovered.

And finally yesterday Redis Labs, a NoSQL vendor, released a NoSQL performance study conducted by Avalon Consulting (which also did the benchmarking that Couchbase won earlier this year). It tested Datastax Cassandra, Couchbase, Aerospike and Redis Labs to determine which is best suited for high volumes of ‘writes,’ relevant for use cases such as real time analytics and Internet of Things data ingestion.

Needless to say, Redis Labs took the top prize.

Why Bother?

To some, these benchmarking tests may seem worthless because whoever sponsors them and sets the parameters seems to win. But Matt Pfeil, DataStax’s co-founder and Chief Customer Officer, sees it differently.

“Those observers unfamiliar with the new NoSQL technologies might consider this benchmark game to appear somewhat ridiculous — with each vendor claiming victory,” he said. “In reality, benchmark reports are a great mechanism that these vendors use to educate the market on the new use cases these new technologies support and the specific ones at which they excel.”

And while DataStax benchmarks against other NoSQL vendors, it’s the use cases supported by legacy technology like Oracle that are being rapidly displaced by new NoSQL technologies that they’re really interested in.

Crowded Race

But DataStax, Couchbase, Redis, and even MongoDB, which says it’s in a position to become the new database default, are hardly the only NoSQL vendors around. Others like Aerospike, Basho (Riak) and even proprietary vendors like MarkLogic have NoSQL products that are also worth mentioning.

When it comes to performance, Peter Goldmacher, Aerospike’s Vice President of strategy and market development, explained that his company’s offering, a memory key value store database built specifically to run in flash provides “enormous performance and volume advantages,” over would-be competitors.

He reasoned that while many databases can run in memory, they aren’t written explicitly to run on SSDs. “We are,” he said. “This means they get 20 percent to 50 percent in improvements while we get 100x improvements.”

Or in simpler terms, Goldmacher compared most NoSQL solutions to engines you might strap onto a bus and call it in memory.

“Our engine is custom fit for a fighter jet,” he said.

There’s also the use case to consider. Goldmacher said many NoSQL products were built as viable alternatives for workloads that have historically run on relational databases like Oracle. On the other hand, companies that are building brand new apps that are doing things for the first time are using Aerospike. He cites the ad tech vertical, which didn’t exist a decade ago as an example.

Speaking of other NoSQL vendors, Aerospike’s market is much smaller, but is growing much faster with the advent of massive data growth and the growing business requirement for immediate decisions.

“They are replacement databases. We are a database for new things,” he said speaking of his would-be competition.

It’s also worth considering that Aerospike may not yet be as popular with developers because it only recently became open source.

Lies, Damn Lies and Benchmarks

Peter Coppola, vice president of product at Basho Technologies, which delivers open source database Riak, isn’t too impressed with benchmarks. He said there are three classes of untruths in the world, “Lies, damn lies and benchmarks.”

That being said, “there is no question that performance is of importance for the modern application workload."

“However, the preternatural fascination that database vendors have with comparing performance numbers is simply a distraction from the real questions that enterprises adopting NoSQL solutions need to ask themselves,” he added.

He pointed out that operations per second is a meaningless metric for many if the database is unavailable, considering that 95 percent of businesses with more than 1,000 employees estimate that they lose more than $100,000 for every hour of downtime.

For more than one in two large businesses, the cost of downtime amounts to more than $300,000 an hour.

Basho’s NoSQL database Riak was designed from the ground-up to prevent losses like that. It was built for Amazon-like performance and uptime without as much as a hiccup. And it rates quite well. Forrester ranked it as a strong performer not far behind Couchbase.

Given that Basho has had some management turmoil of late, that could get even better with the company’s new leadership team.

And though we don’t often hear about MarkLogic in NoSQL conversations, it too is a NoSQL database. While critics call it out for being proprietary versus open source and complain it was built before the days of commodity hardware, JSON popularity, the so called age of big data, the IoT and the like, it was recently raised a $102 million funding round, which pushed its value to more than $1 billion.

When it comes to enterprise worthiness, it’s an older company that has spent years meeting the needs of large companies and even the US Government. In fact, it was selected to power HealthCare.org and NBC’s Saturday Night Live anniversary site.

There's no question this is a time of great opportunity for NoSQL vendors and that competition has the potential to fuel significant innovation. But the game doesn’t have to be winner-take-all, and the rivalries don’t have to look like they did when Microsoft, Oracle and IBM fought for turf.