Search form

Randomness is the Key!

Howto view and analyze your logs on a web page

Submitted by dimaj on Sun, 07/27/2014 - 19:23

Recently I started having some problems with my DD-WRT router. I was having some connection problems and occasional reboots. So, to deal with the problem, I wanted to collect some data from the router in the form of logs. Luckily for me, DD-WRT has a syslogd service which is could send logs to a syslog server on another machine over TCP (or UDP?) connection. Since I know close to nothing in that area, I went to the next best trusted source - Google!

Some of the search results that I've seen referred to something called Logstash, Elasticsearch, and Kibana. To make the long story short, these 3 components make a great centralized log repository with awesome visual data representation and search capabilities.

Got your attention?

Below is a quick guide on how to get up and running with this stack. This guide is goind to setup all 3 components (with appropriate dependencies) on an Ubuntu 14.04 server. Also, for this to be most effective, some pieces should be installed on separate computers. Based on my understanding, Elasticsearch and Kibana should be installed on 1 computer while Logstash should be installed on other coputers to gather logs. Since I wanted to analyze my DD-WRT logs (as well as logs of my current server), I've installed all 3 pieces on a single machine. With that being said, here's the guide.

First thing first, I'm assuming that you have:

Admin rights to your server

Server with Ubuntu 14.04 Desktop / Server

LAMP stack installed

Now with that out of the way, let's get started!

Component Installation

Update the system

Before we begin, update your system by running:

sudo apt-get update
sudo apt-get upgrade

Once that completes, install Java 7 by running:

sudo apt-get install openjdk-7-jre

Installing Logstash

Download latest version of Logstash from their website (For our tutorial, we are going to use version: 1.4.2)

Now, create Apache configuration so that you could access your Kibana instance. In this example, we are going to create a new file: /etc/apache2/sites-available/kibana.conf. Here's a template to get you started: