Machine learning. Artificial Intelligence

Menu

Notes on the Forrester “Wave”

Forrester recently published the Wave™ for Predictive Analytics And Machine Learning Solutions. You can purchase a copy from Forrester for $2,495, or get a free copy here.

When Forrester last delivered this analysis in 2015, they called it the Wave™ for Big Data Predictive Analytics Solutions. So, I guess that Big Data is “out” and machine learning is “in.”

The chart below shows the 2015 Wave™ on the left and the 2017 Wave™ on the right. Green arrows indicate movement in a positive direction, red arrows in a negative direction.

Bob Muenchen reports on the Forrester analysis in the latest update to his article on the popularity of analysis software.

Here are eight notes on the 2017 Wave™.

(1) There are important changes in Forrester’s methodology.

The graphic below shows the measures and weights Forrester used to evaluate vendors in 2015 and 2017.

The 2017 analysis places much more importance on data prep tools, model management, and the solution roadmap. It places correspondingly less weight on business applications, security, and growth rate.

(2) Three vendors are out.

Alteryx, Oracle and Predixion are out.

Alteryx is a data blending tool with the capability to run R scripts. It doesn’t have a native machine learning capability, which explains why Forrester excluded them.

Oracle has native machine learning capabilities, but they are embedded in Oracle Database, and not standalone, as Forrester requires. Oracle also distributes an enhanced R distribution, but that doesn’t count for anything (in Forrester’s eyes.)

IoT vendor Greenwave Systems acquired Predixion last fall, and they are out of the category.

(3) Four new vendors are in.

Dataiku, Domino Data, H2O.ai, and Salford Systems are in the Wave™ this year.

The first three also made it into the 2017 Gartner Magic Quadrant for the first time, so kudos to them. I wrote about them in my analysis of the Gartner report, so no need to say more.

Salford Systems, a company founded in 1984 and the original publisher of CART software for decision trees, was in the Wave™ previously but fell out in 2015. This time, they made it back just in time to be acquired by Minitab. No offense, but who knew Minitab was still around?

— FICO went from Strong Performer to Leader largely on the strength of improved ratings for architecture and data preparation.

— KNIME and RapidMiner leapt into the Leader category. Forrester awarded them much higher ratings in Architecture and Pricing. As far as I know, neither vendor has radically changed its architecture or pricing in the past two years so whatever.

Forrester throws a bit of shade at KNIME:

(KNIME) relies on the community for innovation. However, the result is that sometimes its enterprise features lag larger vendors as the community focuses on other areas, such as new analytical methods.

Shockingly, KNIME’s community cares more about new analytical methods.

— Forrester rated SAP a Leader in 2015 and again in 2017, but with an overall higher rating on the current offering. More on SAP below.

Microsoft improved its rating within the Strong Performer category, but drew this bit of snark:

Microsoft invested heavily in R, thinking it was the go-to language for data science, only to find a few years later that Python was rising in popularity…Microsoft needs to release its bias toward r and go everywhere data scientists want to go.

Mr. Forrester, what are you smoking? For the record, Microsoft distributes Anaconda, supported Python in Visual Studio long before it even thought about supporting R, and offers managed services for both Python and R in Azure.

(5) Angoss is a Leader?

Forrester rates Angoss as a Leader in its 2017 Wave™.

Angoss is ready to be your primary solution. Angoss KnowledgeSEEKER is a must-have for data science teams that wish to use beautiful and comprehensive visual tools to build decision and strategy trees.

I’m curious. When did Forrester start accepting advertorials?

Angoss KnowledgeSEEKER was kind of cool back in 1993. I used to call my colleagues at AT&T to my desk, show them KnowledgeSEEKER, and say “Wow, look at this colorful tree visualization!” They would grunt and go back to writing JCL and SAS code for their mainframe jobs.

In the 27 years since then, I haven’t met a single Angoss user, worked with a client that licensed the software, or encountered it competitively in a sales situation. The company does not register in IDC’s annual assessment of worldwide analytics software revenue.

So, whatever.

(6) SAP is a Leader?

Forrester rates SAP a Leader in the category. Meanwhile, Gartner has demoted the company to Niche status. The difference is attributable to customer satisfaction, which Gartner measures but Forrester doesn’t. Here’s a snippet from the Gartner report:

Similar to last year, reference customers scored SAP in the lowest decile for overall customer satisfaction. For most critical capabilities, SAP (BOPA) received the lowest scores of the vendors in this Magic Quadrant.

Other than that, SAP’s customers are satisfied.

In Forrester’s eyes, however, size matters:

SAP offers comprehensive data science tools to build models, but it is also the biggest enterprise application company on the planet. This puts SAP in a unique position to create tools that allow business users with no data science knowledge to use data-scientist-created models in applications.

You know who likes SAP’s analytic tools? CIOs who want one vendor icon on their PowerPoints.

SAP users do not do so by choice — they work in organizations that decided it was smart to “standardize” on SAP. A large segment of the analytics software industry specializes in pulling data out of SAP and doing something useful with it.

Sure, data tools like SQL. SAP definitely has SQL. It’s a great way to get your data out of SAP. Just write “select *.*” and pipe the results to a flat file. If it’s HANA, a thumb drive will do.

The “distinguished automation tools” include the old KXEN software, acquired a few years ago and rebranded as SAP BusinessObjects Predictive Analytics, (or “SAPBOPA”). I’m not sure how “distinguished” SAP’s automated model training is. The company never seems to win competitive POCs.

Meanwhile, former KXEN customers who aren’t otherwise stuck with SAP stampede for the exits.

(7) Forrester struggles to find nice things to say about IBM.

Forrester rates IBM as a Leader, which is de rigueur if you’re an IT analyst and you want to get invited to those nifty IBM product launches, with the open bar, canapes, and those tasty little sandwiches, served by nice-looking young people in uniforms who only do this part-time, between acting gigs.

SPSS is still the core of IBM’s data science platform

And dropping like a stone. IBM is one of the few vendors in the predictive analytics business with declining software revenue, according to IDC. Only two other vendors in the category did worse: Teradata and Pitney-Bowes.

The good news for IBM: they’re not sinking as fast as Pitney-Bowes.

but IBM is launching projects such as SystemML from its investments in its Spark Technology Center.

Ahem. IBM Research launched SystemML in 2010, five years before the Spark Technology Center opened its doors. IBM couldn’t figure out how to commercialize it — SPSS didn’t want it — so they donated the software to the Apache Software Foundation. There, more than a year later, it languishes in the Incubator, struggling to find users or contributors who are not IBM employees.

Eventually, SystemML may be the greatest piece of software evah. For now, though, it’s a loose piece that doesn’t fit into anyone’s puzzle.

IBM has also introduced the Data Science Experience (DSx) for data science coders, which provides a quick cloud provisioning of open source Jupyter and/or RStudio notebooks with a Spark cluster on the back end to run data pipelines and train models.

DSx is a managed service for Python, R, and Spark that runs in IBM Cloud, everyone’s last choice for a cloud platform. Feature-wise, it competes poorly with other cloud-based data science platforms. Even in IBM Cloud, it has no connection to SPSS, which IBM brands (comically) as “Watson Machine Learning.”

SPSS is a good fit for data scientists who want the productivity afforded by methods encapsulated in operators.

Let’s put that in a Venn diagram.

Hence, “dropping like a stone.”

IBM’s perennial challenge with a comprehensive portfolio is that customers often misunderstand it because there are multiple products that seem to solve the same problem.

It’s not a misunderstanding. IBM’s “comprehensive portfolio” is a hot mess of acquired and rebranded software, plus ill-thought-out new initiatives rushed to market by executives who are desperate to stop the bleeding. Customers understand that better than Forrester does, it seems.

(8) Forrester hearts SAS.

Forrester evaluates SAS the way teens evaluate boy bands. The doe-eyed wooziness begins with the “Product Vendor Information,” where Forrester details the products they assessed for every vendor except SAS:

You cannot license the “SAS Analytics Suite.” SAS’ advanced analytics product page currently lists 32 software products with six different architectures. Why does this matter? Because how you evaluate SAS software depends on which SAS software you’re evaluating.

If you don’t disclose details, you’re blowing smoke.

According to Forrester, SAS has “reimagined” its data science portfolio.

Mr. Forrester, those words, data science — they do not mean what you think they do.

“SAS Visual Suite,” is what SAS calls the combination of two products, SAS Visual Analytics, and SAS Visual Data Mining and Machine Learning. The former runs on the SAS LASR architecture; the latter runs on CAS, the backbone of the new SAS Viya architecture. Two different back ends, and two different programming languages.

They do share the word “visual” in their branding.

If you want to transfer data from LASR Server to CAS, just execute this command:

Be sure to check all eighteen arguments.

Other than that, they are unified.

SAS’ vision for data science is not limited to innovation in tools. it has been quick to jump on new, promising analytical methods across multiple disciplines, such as statistics, econometrics, optimization, machine learning, deep learning, and natural language interaction.

I’ll believe that SAS is serious about deep learning when it supports GPUs.

It recently introduced support for calling SAS analytics from Python, Java, and Lua, leveraging open source data science notebooks.

The three data scientists who use Lua are calling. They want their notebooks back.

A key challenge is that SAS has a target on its back by the open source zealots that summarily and wrongly dismiss SAS as old school.

5 comments

Hello Thomas – Snark is first rate as usual – love how you mock SAS as old school in almost the same breath as you mock lua as too new to care about :-). One small point about SAS Visual Analytics. the 7.x generation is/was backended by our LASR server and the current 8.x generation is backended by the newer CAS server. the API reference (important to some folks) you posted is for that set of folks that want to interoperate between the two server generations. (Paul, worked @ SAS for 35yrs)

The folks at Pontifical Catholic University of Rio de Janeiro introduced Lua in 1993, so it’s just as “new” as the SAS Multi-Vendor Architecture. 🙂

Appreciate the clarification about VA and CAS. How are the migrations going? I spoke to an architect at one of the top U.S. banks two weeks ago — he said SAS could not produce a single reference customer in production on the latest version. Admittedly, it’s only six months since the Viya architecture went GA, but still….