David Menninger's Analyst Perspectives

Splunk Delivers Insights from Machines

I recently attended .conf2016, Splunk’s seventh annual user conference. Splunk created the market for analyzing machine data (shorthand for machine-generated data), which consists of log files and event data from various types of systems and devices. Our big data analytics benchmark research shows that these are two of the most common sources of big data that organizations analyze. This market has proven to be fertile ground for Splunk, growing steadily with revenues more than doubling over the previous two fiscal years. Machine data is also the backbone for the Internet of Things (IoT) and operational intelligence, which form the basis of forthcoming benchmark research from Ventana Research.

At the event, Splunk announced general availability of Splunk Cloud and Splunk Enterprise 6.5. The company also announced new versions of Splunk IT Service Intelligence, Splunk Enterprise Security and Splunk User Behavior Analytics. These new versions incorporate machine learning capabilities to help organizations analyze the massive volumes of machine data they collect with more advanced analytics and in a more automated manner. Machine learning has become a hot topic lately; it was also a popular subject at Strata+Hadoop World, as I wrote recently.

The machine learning capabilities, which arose in part from Splunk’s July 2015 acquisition of Caspida, have been added to Splunk Cloud and Splunk Enterprise 6.5. Machine learning is a method used to develop predictive analytics without explicitly programming the models. In effect the algorithms are designed to sift through the data, learn from it and make predictions. With Version 6.5 Splunk also has simplified its data preparation capabilities and enhanced its user interface to appeal to more types of users. The company also offers tighter integration with Hadoop in this version. Storing historical data in Hadoop can help lower costs, and the Hadoop data can be combined with data in Splunk Enterprise using the Splunk query capability for a single unified interface.

Splunk IT Service Intelligence (ITSI), an application built on the Splunk platform, provides a view of how critical IT services are operating as well as an environment in which to investigate and triage incidents when they occur. The latest release of ITSI, 2.4, includes machine learning capabilities to perform anomaly detection, identifying unusual system activity to help prevent outages and service degradations. The system can learn what the pattern of normal operations looks like and then establish thresholds for alerts that adapt to cyclical changes in usage. Adaptive alerts help reduce “alert fatigue” when so many alerts are issued that they overwhelm the recipients.

Splunk Enterprise Security (ES), a security information and event management (SIEM) application, provides real-time monitoring of security threats and an environment to support incident response teams. Splunk ES 4.5, the latest release, provides a similar adaptive alerting feature based on machine learning as described above. ES 4.5 now includes the Glass Tables feature that has been available in ITSI, which allows users to create custom visualizations and KPIs. Splunk User Behavior Analytics (UBA) complements ES by analyzing longer periods of history to create a profile of normal user behavior and comparing it with peers to provide more advanced detection of security threats. UBA 3.0 incorporates more than 40 machine learning models, which cover a combination of streaming and batch analytic scenarios. Splunk in 2015 received the Technology Innovation Award for CIO for its innovation in advancing cybersecurity through these products.

Splunk has followed a unique path. While a pioneer in the big data market, it built its products on a proprietary big data architecture rather than open source technologies as others did. In recent releases, however, it has broadened its support for Hadoop. Splunk focused on one subset of big data – machine data – and based much of its user interface around search. Rather than expand into the horizontal business intelligence market the company has chosen to tackle the IT service market and the SIEM market. This focus appears to have been successful so far. It’s hard to argue with its success. If you are looking for a way to manage and analyze the machine data in your organization, including IT service applications or enterprise security, I recommend you consider the offerings from Splunk.

Related Research

David is responsible for the overall research direction of data, information and analytics technologies at Ventana Research covering major areas including Analytics, Big Data, Business Intelligence and Information Management along with the additional specific research categories including Information Applications, IT Performance Management, Location Intelligence, Operational Intelligence and IoT, and Data Science. David is also responsible for examining the role of cloud computing, collaboration and mobile technologies as they affect these areas. David brings to Ventana Research over twenty-five years of experience, through which he has marketed and brought to market some of the leading edge technologies for helping organizations analyze data to support a range of action-taking and decision-making processes. Prior to joining Ventana Research, David was the Head of Business Development & Strategy at Pivotal a division of EMC, VP of Marketing and Product Management at Vertica Systems, VP of Marketing and Product Management at Oracle, Applix, InforSense and IRI Software. David earned his MS in Business from Bentley University and a BS in Economics from University of Pennsylvania.

Analyst Perspective Policy

Ventana Research’s Analyst Perspectives are fact-based analysis and guidance on business, industry and technology vendor trends. Each Analyst Perspective presents the view of the analyst who is an established subject matter expert on new developments, business and technology trends, findings from our research, or best practice insights.

Each is prepared and reviewed in accordance with Ventana Research’s strict standards for accuracy and objectivity and reviewed to ensure it delivers reliable and actionable insights. It is reviewed and edited by research management and is approved by the Chief Research Officer; no individual or organization outside of Ventana Research reviews any Analyst Perspective before it is published. If you have any issue with an Analyst Perspective, please email them to ChiefResearchOfficer@ventanaresearch.com