Logging overview

You can use Splunk Enterprise, Splunk Cloud, Splunk Light, or Splunk Cloud
Light (collectively, "Splunk software") immediately in your everyday development activities to
help improve the quality of your applications and reduce the time it takes to
resolve issues. With Splunk software you can quickly and easily search your log files. You
can also make some slight code changes to improve the readability of your logs across
the entire organization.

Logs can be a hassle to deal with: they don't have a real structure, you might
not know what's in them if you didn't create them yourself, and the sheer number
of files and types of logs can be overwhelming. On the other hand, logs are really
useful: they contain a gold mine of analytic information, they can help you find
problems, and they can tell you about your IT infrastructure, the behavior of your
users, and identify potential attackers.

Chances are, you're already using logs for debugging purposes. But you can do
more with your log files. Keep reading to learn how to get the most from your logs
and how to set them up right. The results are worth it.

What you can learn from logs

There are two types of logging:

Logging for debugging. As an application developer, you can use Splunk
software to
look at your own application logs for debugging rather than hunting and pecking
through files. Executives and others might be interested in viewing information
from debugging logs to identify trends in system stability, categorize issues,
and more.

Semantic logging. You can also log "semantic" level events, which
are events that record the state of business processes (web clicks, financial
trades, cell phone connections and dropped calls, audit trails, and so on).

Typically, IT organizations develop systems that focus on specific silos of technologies,
functions, people, or departments. Individually, these silos provide a limited view.
But they all generate data—data that is non-standard, unstructured, high-volume,
and growing every millisecond of every day. Whatever you're logging, every environment
has some set of IT data to leverage:

Logs contain a record of customer activity and behavior, product and service
usage, and transactions.

Logs contain a record of activity for IT components, such as applications,
servers and network devices.

Often, companies must write their own applications from scratch to perform statistical
analysis on the data contained in these events. Similarly, companies build dashboards
by hand to display that analysis for historical comparisons, future projections,
performance tuning, billing, security, and so on.

Splunk software doesn't care about the format or schema of your data—queries and
searches can be ad-hoc, and your data can come from any textual source. With Splunk
software,
you can use the logs generated from your application to achieve the same powerful
analytics without building special-purpose software. Logging semantic data in addition
to debugging data can significantly enrich virtually all applications.

This pseudo-code logs the purchase ID for each action. With the two events
generated from the above pseudo-code, you could learn a lot about your purchase
transactions. You could:

Graph your purchase volume by hour, by day, and by month.

Find out how long purchases take during different times of the day and days
of the week.

Find out how long purchases are taking now compared to last month.

Find out whether systems are getting slower.

Find out how many purchases are failing, and graph these failures over time.

Find out which specific purchases are failing.

Each of these items is a simple search in Splunk software that you can save, schedule,
and share by e-mail. And there is no need to architect a complex solution—the
more data you log, the more you get out of it. It's really that simple. Use Splunk
software instead of complex and inflexible architectures like RDBMS/SQL.

It's important to set up your logging in the right way to get the
maximum benefit with your application. Start by thinking about how other
people will view your logs. Events might get separated from the logs. Certain events
might get copied and passed along to a variety of people inside and outside of an
IT organization. Without using robust logging techniques, you can lose meaningful
data, making it harder to derive insight from the logs. The logging best practices
that follow can help make your debugging logs more relevant and powerful for you
and others.

Questions?

We use our own and third-party cookies to provide you with a great online experience. We also use these cookies to improve our products and services, support our marketing campaigns, and advertise to you on our website and other websites. Some cookies may continue to collect information after you have left our website.
Learn more (including how to update your settings) here »