However, as markets go, this one is incredibly competitive. Which makes it interesting to analyse in terms of how 1) how customers are using and shaping the technology, and 2) how the top vendors are shifting their approach to market.

These two points are inextricably linked, as customers demand more from the NoSQL providers, pushing them in certain directions. I’ve always been of the view that whilst NoSQL databases are all well and good, the providers are going to have to do more to ensure that in the medium to long-term they don’t just become another commodity database – particularly as the technology is open source.

And what do I mean by ‘do more’? It has always made sense to me that the providers of NoSQL tech should move up the stack and eventually become more of a traditional software business, providing suites for industry based on NoSQL technology.

It’s something I’ve put to all the main providers out there (MongoDB, Couchbase, Basho), but DataStax is the first to give me a clear confirmation that that is the direction it is headed.

DSE 5.0

I got the chance to sit down with DataStax co-founder and Chief Customer Officer, Matt Pfeil, in London this week, where he gave me the rundown on the company’s latest announcements and some insights into it’s future direction.

DataStax has just announced its Enterprise (DSE) 5.0 offering, which is the latest version of its Cassandra-based multi-model database solution. The key highlights of the latest release include:

DataStax Enterprise Graph – a scalable real-time graph database built for cloud applications, which allows for the management of complex data with many relationships. This is based on TitanDB and is the result of the company’s acquisition of Aurelius (but more on this later).

DSE Advanced Replication – specifically designed for support IoT and retail applications, which need to replicate data within hub-and-spoke and similar topologies.

Advanced server automation – simplifies data management via DSE tiered storage, which automatically moves data to its appropriate storage tier based on age and where the data is hot/cold. This makes data storage more cost effective.

Advanced analytics and search – includes an updated version of Apache Spark, which enables increased speed for streaming data, enhanced connectivity with external Spark clusters, and faster enterprise search operations with Apache Solr.

Matt Pfeil, DataStax

Pfeil said that one of the most exciting aspects of the release was the inclusion of the graph technology. He believes it will allow customers to better use data to power their business, something Pfeil argues has been ignored by many to date. He said:

I don’t think graph is quite as understood, because it’s newer. But if you look at general use cases, it’s all about relationships. The question then is, what kind of problems in the world really need relationships? An extremely obvious use case would be a social network of some sort. That may not be a giant monetisation aspect for us. But the less obvious ones for us would be like fraud detection, recommendation engines.

We saw people building graph-like technology in their application and people were not doing it extremely well, because it’s not their area of focus. Or they were just spending a lot of time on it and they didn’t want to.

I would venture to say that people aren’t using data strongly as part of their business operations, whether that’s just underlying Cassandra or a graph solution. Their competition will provide a better solution and they will go out of business. I think the data aspect of this cannot be understated in terms of how important it is for day to day success.

Moving up the stack

Graph is one of the first indications from DataStax that it is thinking more broadly about the capabilities it can build out on top of the NoSQL technology. And even though graph is still technically quite low down the stack in the grand scheme of things, we are certainly seeing an upwards shift in direction.

And this is being guided by the customer use case, according to Pfeil. He said:

One of cool things about being in our spot, related to core Cassandra as a general purpose database, is that we see ways people are using our technology and you can look for patterns. And when we identify patterns we can subsequently provide higher level functionality.

Most people don’t want to work on databases, they want to work on the core business. So we can actually create higher level value for the customer by building up the stack. Today we are in the database business, but I could easily see a decade or two from now providing much higher level business functionality. But get there incrementally over time.

Pfeil said customers should think about DataStax as Cassandra at the core, with it then building out a rainbow of layered business capabilities as it matures. He said:

So we look for those patterns and build out layer by layer. So we started with Cassandra back in the day. We have added over the years search, analytics and security, which is still pretty rudimentary. The next level us is actually this, so graph, which is higher level.

If you look at the tier above that, you get into things like general purpose recommendation engines. Or general purpose fraud dedication. You have to do that layer by layer, you never want to lose the core. Because if you do, you lose visibility into what people are doing with it.

And what about ten years from now?

I think we want to be a data driven software company…I think at that point we would have business level solutions.

Unless we start to see some consolidation in the market, of course (which is inevitable). He added:

I don’t think anyone sees acquisition happening until it happens. We’re not building something to get acquired. Everything has a price, I’ll put it that way. But in terms of product vision, that’s not what we are trying to achieve.

Future challenges

DataStax’s customer base includes the likes of Netflix, eBay, Target and UBS – big names, all using NoSQL to provide internet-age applications to their customers. However, despite the big names, DataStax cannot become complacent about its position in the market.

As I noted above, the NoSQL market is becoming increasingly competitive and DataStax faces stiff competition from the likes of Basho, Couchbase and MongoDB. All of which have faced similar degrees of success.

So who does Pfeil see DataStax’s future challenger being? As it moves further up the stack into more rounded business solutions, this will likely see DataStax competing in markets that it doesn’t yet compete in today. He said:

There are only a few of the so-called NoSQL players that have raised over $100 million, so I don’t think there is going to be new challengers on that front. If you’re just getting started, why would an investor give you $3 million to take on a company that’s got $200 million. The odds of success are small.

In terms of what might be a challenger five years from now, as we move up those tiers, it will be really intriguing for us to look at as we get to a certain higher level functionality, there might be an incumbent in that space. The thing I like about that is, is that that would likely be someone’s entire company, whereas for us it would be one aspect.

But at the same time it will be a whole new attle with them on functionality, so we will have to choose our battles really carefully there.”

I would say that our largest challenge right now is that you have to maintain a company expectation around growth, whilst simultaneously managing the skills gap. We were helping the skills gap on our existing technology, but we are going to be introducing new.

So we have to continue both of those in parallel, whilst the company is continuing to grow. There are more moving parts, so maintaining that takes a lot of discipline internally.

Whenever I sit down with a fast growing company, I also always ask if they feel like they aren’t yet providing customers with something that they’re seeing demand for. This usually provides at least some indication about future roadmaps or changes that the company may implement in the short to medium term.

When I put this question to Pfeil, he made an interesting observation around DataStax customers’ use of cloud – along with an interesting breakdown of current use. He said:

I would say that the only thing that might start to catch our attention a bit, is that historically most of our customers because of their scale have been using their own data centres. We are seeing more adoption in cloud. For example, by customer count roughly half of our customers are running in the public cloud. But by node count, or by dollar, which has a direct correlation, it’s less than 20%.

In other words the bigger customers are running their own data centre. We have historically to date attempted to satisfy the as-a-service offering through partners, but it’s lightweight partnerships. But I’d imagine we’d spend more time there trying to do it deeper through partnerships or will get involved some way shape or form directly.

It’s just an area where I think as-a-service is going to be more in demand for our offering.

My take

It’s always good to catch up with Pfeil. And whilst DataStax doesn’t have the aggressive marketing/sales approach that MongoDB does, it manages to lay out a convincing and cogent strategy to its end users.

The fact that DataStax is now talking about moving higher up the stack and is starting to think in terms of ‘business solutions’, I think will stand it in good stead. If it doesn’t do this, it risks becoming a commoditised database solution as the market matures. I’m sure DataStax’s competitors will be taking note.

One word of warning though: as DataStax starts to think about its opportunity further up the stack, it needs to consider it’s approach to market. Currently the company (along with its competitors) have a very technical audience and its customers are often developers. And it’s approach is very technical If it’s going to start thinking about business solutions, that’s a very different sell and it needs to be prepared to shift gears.