Machine learning. Artificial Intelligence

Menu

Notes on Gartner’s 2019 MQ

Gartner recently published its 2019 Magic Quadrant for Data Science and Machine Learning Platforms.You can secure a copy of the report from Gartner if you are a client, or read it for free here, courtesy of DataRobot. (Registration required.)

Here’s how Gartner positioned the 17 vendors that made it into the MQ:

Back in October, I wrote my predictions for the 2019 MQ. You can read them here. FWIW, it looks like I got eleven vendors right and five vendors wrong – two of them VERY wrong.

Every year, people ask the same question: why does Gartner rate KNIME and RapidMiner as leaders? Those two companies aren’t setting the world on fire commercially.

The simple answer is that they offer fully-featured products and have very satisfied users. That formula seems to be lost on some other vendors, including IBM, Microsoft, and SAP.

Here are my comments on selected individual vendors, in alphabetical order. This year I’m going to try to keep my comments positive. My grandmother used to say that if you can’t say something nice about someone, say nothing.

A reminder that DataRobot is my employer and my bias is obvious. Also, this is my personal blog, and the opinions expressed here are my own. And they are awesome.

The company acquired Yhat in 2017. Other than rebranding the software as Alteryx Promote, Alteryx hasn’t done much with Yhat. That’s not too surprising. When your main product runs only on Windows and you acquire a product that runs only on Linux, it may take some time to integrate the two.

Most of the Yhat people are gone, so don’t expect Alteryx to make any progress in machine learning.

Amazon Web Services

AWS isn’t in the Magic Quadrant. This is surprising. Amazon SageMaker and Amazon Forecast are better than anything Google has to offer, and Google’s in the MQ.

Big Squid

Big Squid isn’t in the MQ either. Gartner gives Big Squid an Honorable Mention in 2019 for the second year in a row. That’s like getting named to a list of promising AAA players for two consecutive years. At some point, people will wonder whether you can hit a major league curveball.

Cloudera

My friends and former colleagues at Cloudera took my advice and chose not to participate in the Magic Quadrant. Good move. They would have scored among the Niche Players, as data platforms always do. Gartner rightly figures that data scientists must work with diverse data platforms. Tools that integrate with just one data platform are Niche Players by definition.

Databricks

Kudos to the Databricks people, who keep powering on to new accomplishments. Investors agree.

Dataiku

Dataiku improved markedly on Gartner’s Ability to Execute. The company moved from the bottom of the Visionaries quadrant to the Challengers quadrant, within spitting distance of the Leaders quadrant.

This illustrates a maxim of working with Gartner: if Gartner points out a weakness in your offering and you do something about it, Gartner rewards you. In the 2018 MQ, Gartner cautioned readers that Dataiku customers had issues with model deployment. Dataiku addressed that with some enhancements in Releases 4 and 5.

Now, Gartner advises Dataiku that if it adds streaming analytics it can be a Leader. That’s terrible advice. Streaming analytics is the most overhyped thing in data science. Even Gartner thinks it’s past the Peak of Inflated Expectations in its 2018 Hype Cycle for Data Science. IT analysts who haven’t a clue about machine learning get all warm and fuzzy over streaming analytics, but actual use cases are few and far between.

As a competitor, I encourage Dataiku to invest a lot of time and energy investing in streaming analytics.

DataScience.com/Oracle

DataScience.com didn’t quite make it into the MQ last year. They had a booth at Strata that was bigger than my house, but that wasn’t enough to make the cut.

Oracle bought the company last June and rebranded the product as Data Science Cloud. Forrester promptly rated Oracle as a Leader in the Notebook-based data science platforms category. Oops. By the time Forrester published its report, Oracle had pulled the software out of production.

Forrester noted in its scorecard that the company had no customers. Ordinarily, having customers is a prerequisite for market leadership.

I’m not sure why Oracle bothered to buy DataScience.com. In the time needed to port the software to Oracle Cloud, they could have built something from scratch.

Trying to stay positive here. Grandma never had to deal with Oracle.

Domino Data Lab

Ouch. Someone at Gartner doesn’t like Nick Elprin. The 2019 report on Domino is a dumpster fire.

It’s funny because Forrester rated Domino as the leader in Notebook-based data science platforms. It really does not make sense to lump tools for coders together with tools for clickers.

Gartner can argue until the sun goes down that “citizen data scientists” will eat the world. Maybe that’s true, and maybe it isn’t true. People said that SPSS Modeler and SAS Enterprise Miner would democratize advanced analytics. Funny how that never really happened.

There is a real demand for software that supports data scientists who write code. That’s why Forrester stopped lumping all of the data science products together.

Google

OK, I said I was going to keep my comments positive. I lied. Google has nothing.

Yeah, I know. TensorFlow. Fine. It’s one of many open source packages that data scientists can use. And you can use it anywhere, you don’t have to use Google’s commercial service.

Google deprecated its prediction API last April. Meanwhile, the company has a slew of services in beta, like Cloud AutoML, Kubeflow Pipeline, and BigQueryML. Maybe those services will amount to something someday. If they do, I’ll change my opinion.

By the way — Cloud AutoML supports vision, speech, and translation analytics. It doesn’t do the most common classification or regression use cases. And Google Cloud DataLab is one of the weaker notebook-based services.

If you want a cloud-based platform for data science and machine learning, check out AWS and Microsoft Azure.

H2O.ai

H2O.ai was a Leader in the 2018 Magic Quadrant. The company heavily promoted that position throughout the year.

So it’s gotta burn that the company fell out of the Leaders quadrant in the latest report. What Gartner giveth, Gartner taketh away.

For the second year in a row, Gartner cites Deep Water as an example of H2O.ai innovation. Someone should tell Gartner that H2O.ai deprecated that product more than a year ago.

IBM

Grandma didn’t read IBM press releases.

IBM is really good at cherry-picking results to present a rosy picture.

The Rams won the Super Bowl 3-0 on plays where the Patriots did not score.

IBM fell out of the Leaders quadrant in 2018. IBM executives figured they could climb back in with IBM Watson Studio, a quodlibet of new and rebranded software. Instead, the company sank deeper into the mire.

Gartner gives Watson Studio pretty good marks. Forrester likes the product enough to rate IBM a Leader in its Wave report. Unfortunately for IBM, Watson Studio is just one part of a picture that also includes SPSS.

Watson Studio could be the greatest data science and machine learning software ever. It doesn’t matter. Considering the way IBM treats SPSS customers, will any of them even think of buying something new from IBM?

Microsoft

Microsoft was a Niche Player in this category when all it had to offer was machine learning embedded in SQL Server. Rightly so. Nobody used that shit. Sorry, Grandma.

Then, when the company launched Azure Machine Learning Studio it leaped into the Visionaries quadrant. AMLS is a nice service. I used it in a class for data science wannabes. They were all building predictive models in an hour.

But Microsoft hasn’t moved significantly in three years. Meanwhile, it rolls out products and services that are oddly disconnected from one another. It’s as if development teams work in silos, and nobody is in charge.

Which might explain why Joseph Sirosh is no longer in charge.

Pitney Bowes

Remember Quadstone? After a few mergers and acquisitions, what’s left of that software ended up at Pitney Bowes.

I’d say that Pitney Bowes is just mailing it in at this point, but they didn’t even make Honorable Mention this year. So I guess they’ve stopped mailing it in.

SAS

Folks at SAS joke that the most dangerous place on campus is the exit from the Engineering parking lot at 4:45 in the afternoon. In fact, the most dangerous place at SAS is the Chief Sales Officer role. Nick Lisi is the latest CSO to exit.

The previous CSOs weren’t losers. The problem is that Jim Goodnight thinks that SAS sellers should grow the top line at double digits, like the rest of the industry. Instead, the company can barely make last year’s numbers.

Of course, when last year’s numbers were $3 billion, just matching them is an accomplishment. Still, Goodnight wants more. He’s the boss, so he gets to set the goals, even if they are ridiculous.

You may have heard the story about the CEO at a leading pet food company. Unhappy with sales growth, he brought in McKinsey to investigate.

Three weeks into the engagement, the McKinsey partner met with the CEO.

“I have some bad news,” said the partner.

The CEO gave him a worried look. “Bad news?”

“Yes. There’s a problem with the dog food.”

“What’s the problem?” demanded the CEO.

“Dogs don’t like it.”

Goodnight should hug Sales for just hanging on. An executive at a very large SAS customer says his organization has a strategic goal to reduce SAS spending by 30%.

A strategic goal, mind you. Not a wish. Not something nice to do. A strategic goal. Companies apply brainpower to meeting their strategic goals. They hold people accountable.

It’s kind of hard to grow the top line when your biggest customers want to cut your legs off.

Teradata

I predicted that Teradata would exit the MQ entirely. That was a no-brainer once the company deprecated Aster, the product it submitted in the last few MQs.

Gartner’s kind of strict about that. If you don’t have a product, you don’t get to be in the MQ. It’s like baseball. They don’t let you step up to the plate unless you have a bat.

The recently launched Vantageproduct looks to me like recycled Aster assets and open source software running in containers. It’s Shakespearean: much ado about nothing.

Here’s a Bye-Ku for Teradata:

Teradata says

It has machine learning, but

It doesn’t, no, no.

TIBCO

The first reaction of most readers when they see TIBCO in the Leaders quadrant is “who the fuck is TIBCO and why are they in this MQ?”

TIBCO isn’t a complete stranger to the world of advanced analytics. The company acquired Insightful in 2008. Insightful had a rudimentary tool for statistics, and it also owned S-Plus, a commercial version of S, a predecessor to R. TIBCO rolled the statistical functions into BI/Viz tool Spotfire and repackaged the S-Plus assets as a runtime engine for R.

That’s not quite as ridiculous at it sounds. R scripts run on S-Plus. At least some of them do. With exceptions.

Hey, who cares if it messes up your date/time math. Who’s going to know?

In 2017, TIBCO acquired Statistica and Alpine. Statistica is a piece of software developed in the 1980s by StatSoft as a cheap desktop alternative to SAS. The company did modestly well for many years. There are a lot of folks out there who want to run a t-test now and then but don’t want to shell out $$ for SAS. But the owners wanted to cash in and retire, so they sold the company to Dell. Dell sold the company to private equity investors, who sold it in turn to TIBCO.

Lots of transaction fees for investment bankers. One wonders how the StatSoft people in Tulsa fared.

Longtime readers of this blog will be familiar with Alpine. Alpine started life in the same incubator as Greenplum. When Greenplum sold itself to EMC in 2010, it left the machine learning assets behind, like that guy who left his baby on the bus. Anderson Wong, Steve Hillion, and Yi-Ling Chen secured an “A” round and founded Alpine in 2011. The company landed a “B” round in 2013, and then…nothing. It seems that the Total Addressable Market for a drag-and-drop tool that works with Greenplum isn’t large.

TIBCO rebranded Alpine as Spotfire Data Science. Gartner seems impressed that TIBCO has integrated all of this stuff into something coherent. Color me skeptical. The word “Spotfire” does not appear in Statistica documentation, nor does the word “Statistica” appear in Spotfire documentation.

It looks to me like the two products are integrated in the sense that both carry the TIBCO brand.

TIBCO’s been offering Statistica to existing Spotfire customers for free. That seems like a fair price.

Just as an aside, Tibco also purchased Composite Software’s data virtualization stack from Cisco. They are on the move into the data space. Plans for ML? Who knows but DV is important. Look into it if time permits (curious on your thoughts).

Teradata is a cluster. I doubt they survive on their own. They are laying off people quarterly.

DV has performance issues at volume. First hand experience. If they can improve the performance, phenomenal technology. I personally love DV. Integrating ML across disparate systems using DV as a buffer is a viable concept even if a bit challenging right now. It makes data common w/o tying everything together.

I think TD and similar vendors days are numbered. Everything is cloud in the future and microservers.

Great analysis. Gartner and similar companies are like Yellow Pages. I can’t wait for more disruptors ( sounds like that could be you ) to negate a need for Gartner or research firms. It’s so 1990-2000s.

Cyclists riding near the SAS campus exits at that time do so at their own peril. The stretch of road to the exits was the most harrowing part of my evening bike ride. Employees will darn near clip, brush, or run right over you without a second thought. I’ve experienced it first hand. It’s frightening, and not a joke!

(1) SAS has saturated the market for its legacy tools. Every organization that will ever use SAS already uses it. (That was true in the early 2000s, FWIW.) SAS continued to grow for some years through account expansion and new products, but many customers feel that they already spend too much on SAS and want to reduce their footprint. Cloud computing and open data science contribute to that trend. At one time, SAS was widely used in colleges and universities. Now new grads entering the workforce use open data science tools.

(2) Slash the workforce to reduce costs. Sell the farm, the art, the European castles, and the rock collection. Sunset legacy software and Goodnight’s vanity projects. Donate the SAS programming languages to open source under independent governance, such as Apache. Continue to sell support contracts for the open source bits. Continue to offer the higher-end software under commercial licenses. Fire most of the leadership and hire executives who have worked somewhere other than SAS.

(3) I’ve heard that it works, from three different customers. I’ve also heard from a fourth customer that they’ve spent a year trying to get it to work and they’re still struggling with it. A SAS insider tells me that there are no more than 100 companies that use it. That seems low, and it’s unconfirmed, but that’s the only number I’ve heard. Many companies simply don’t care. They don’t plan to expand their SAS footprint. Viya could be the greatest product in the history of products, they’re not interested.

Regarding (2) above, SAS 76.2 was put into the public domain. It’s argued by some and by extension the language, grammar, keywords, etc… are likely included in that. See the court findings (specifically section 827) https://tinyurl.com/y6c332dq using Google Scholar. For those unaware, World Programming who makes a SAS Language compatible software product has won numerous court cases in both the UK, EU and US regarding SAS copyrights.