What Is Machine Data?

Digital Exhaust. Log Files. Time-Series Data. Big Data.

Whatever you call it, machine data is one of the most underused and undervalued assets of any organization. But some of the most important insights that you can gain—across IT and the business—are hidden in this data: where things went wrong, how to optimize the customer experience, the fingerprints of fraud. All of these insights can be found in the machine data that’s generated by the normal operations of your organization.

Machine data is valuable because it contains a definitive record of all the activity and behavior of your customers, users, transactions, applications, servers, networks and mobile devices. It includes configurations, data from APIs, message queues, change events, the output of diagnostic commands, call detail records and sensor data from industrial systems, and more.

The challenge with leveraging machine data is that it comes in a dizzying array of unpredictable formats, and traditional monitoring and analysis tools weren’t designed for the variety, velocity, volume or variability of this data. This is where Splunk comes in.

The Splunk platform uses machine data—the digital exhaust created by the systems, technologies and infrastructure powering modern businesses—to address big data, IT operations, security and analytics use cases. The insights gained from machine data can support any number of use cases across an organization and can also be enriched with data from other sources. The enterprise machine data fabric shares and provides access to machine data across the organization to facilitate these insights. It’s what we call Operational Intelligence.

Software Download

The Essential Guide to Machine Data

Watch this video introduction to the most common use cases around machine data. Splunk's Doug May breaks it all down in this nifty lightboard demo. Oh, and don't ask how Doug can write backwards. It's magic!

Machine Data Sources

Every environment has its own unique footprint of machine data. Here are a few examples.

Data Type

Use Cases

Examples

Amazon Web Services

Security & Compliance, IT Operations

Data from AWS can support service monitoring, alarms and a dashboards for metrics, and can also track security-relevant activities, such as login and logout events.

APM Tool Logs

Security & Compliance, IT Operations

APM tool logs can provide end-to-end measurement of complex, multi-tier applications, and be used to perform post-hoc forensic analytics on security incidents that span multiple systems.

Authentication

Security & Compliance, IT Operations, Application Delivery

Authentication data can help identify users that are struggling to log in to applications and provide insight into potentially anomalous behaviors, such as activities from different locations within a specified time period.

Firewall

Security & Compliance, IT Operations

Firewall data can provide visibility into blocked traffic in case an application is having communication problems. It can also be used to help identify traffic to malicious and unknown domains.

Industrial Control Systems (ICS)

Security & Compliance, Internet of Things, Business Analytics

ICS data provides visibility into the uptime and availability of critical assets, and can play a major role in identifying when these systems have fallen victim to malicious activity.

Medical Devices

Security & Compliance, Internet of Things, Business Analytics

Medical device data can support patient monitoring and provide insights to optimize patient care. It can also help identify compromised protected health information.

Network Protocols

Security & Compliance, IT Operations

Network protocol data can provide visibility into the network's role in overall availability and performance of critical services. It's also an important source for identifying advanced persistent threats.

Sensor Data

Security & Compliance, IT Operations, Internet of Things

Sensor data can provide visibility into system performance and support compliance reporting of devices. It can also be used to proactively identify systems that require maintenance.

System Logs

Security & Compliance, IT Operations

System logs are key to troubleshooting system problems and can be used to alert security teams to network attacks, a security breach or compromised software.

Web Server

Security & Compliance, IT Operations, Business Analytics

Web logs are critical in debugging web application and server problems, and can also be used to detect attacks, such as SQL injections.