Tag: Graylog2

In this post we will do a presentation of Graylog2 features, components and installation details on Centos 5.

Graylog2 features and components presentation

Graylog2 is a free and open source log management solution that provide you a centralized repository and access to all your infrastructures logs. All the logs are stored in MongoDB, a scalable and high-performance database.

Graylog2 is composed of a server written in Java that will accept Syslog messages via TCP, UDP or AMQP (Advanced Message Queuing Protocol). AMPQ is an open standard for Messaging Middleware that allow different platforms in different languages to send messages to one another. Graylog2 is also using Drools Expert to evaluate all incoming messages against user defined rule file.

Graylog2 Web interface will allow you to search through the logs, filter them, blacklist out certain messages and create “streams“. An unlimited number of user can access the Web interface how will be able only to read defined and subscribed “streams“. Graylog2 Web interface also provide a way to use Nagios to check of the number of new log messages exceeds a given maximum.

Graylog2 “streams” are saved searches that permit you to quickly access to an overview of specifics occurrences. You can forward your “streams” to other endpoints through UDP Syslog, GELF or to Loggly, a cloud log management solution. GELF (Graylog Extended Log Format) will allow you to by pass Syslog limitations (message length, structure, timeouts, connection troubles) for your applications and servers. “Streams” will also allow you to send alarms when the number of new message reaching a given maximum during a given period. All users who subscribed to the “stream alarms” or to the “stream” will get an email alarm.

Graylog2 Centos 5 installation

In order to have a complete and functional Graylog2 log management solution we have to install three main components, MongoDB, graylog2-server and graylog2-web-interface.

MongoDB database installation

MongoDB propose to Centos and Fedora users yum-installable RPM packages for x86 and x86_64 platforms. “mongo-10gen” (mongodb client) and “mongo-10gen-server” (mongodb server) are available through the 10gen MongoDB repository. Just follow the “Centos and Fedora Packages” documentation to allow you server to install these packages. Then simply execute the following command to install MongoDB server and client.

$ sudo yum install mongo-10gen-server

MongoDB configuration file is located in “/etc/mongod.conf” and the associated sysconfig file is in “/etc/sysconfig/mongod“. When started MongoDB will run under mongod user and group.

First edit the MongoDB configuration file and change “nohttpinterface = false” to “nohttpinterface = false“. Then start MongoDB server with the following command.

$ sudo /etc/init.d/mongod start

Now we will create a user and password to allow Graylog2 to connect to MongoDB server. The database will directly be created during the user creation, and the database will be stored in “/var/lib/mongo“. To connect you to MongoDB server you have only to run the “mongo” client.

$ sudo mongo
> use graylog2
> db.addUser(“login”, “password”)

We have now a user (login) created, with his associated password, for database graylog2. If we wish to perform further operations we need to execute the following command.

> db.auth(“login”, “password”)

We can view existing users for the database with the following command.

We need now to configure the MongoDB server default listener port (27017/TCP). Just uncomment “port = 27017” line in the MongoDB configuration file. MongoDB will listen on the loopback (127.0.0.1). Also we need to turn on security for authentication by uncommenting the “auth = true” line.

Now restart MongoDB with the following command.

$ sudo /etc/init.d/mongod restart

Graylog2 server installation

Graylog2 server require to install openjdk.

$ sudo yum install openjdk

Download Graylog2 server from Github and adapt the following commands to your need.

Also configure, in the same configuration file, the Syslog server listener port and protocol. By default the Syslog server is listening on 514/UDP.

Now start Graylog2 server with the following command.

$ sudo cd /opt/graylog2/bin/
$ sudo ./graylog2ctl start

To stop graylog2 server execute the following command.

$ sudo ./graylog2ctl stop

Graylog2 Web interface installation

Graylog2 Web interface is running under Ruby, so we first need to install the latest version of Ruby. Please remove all your previous Ruby installation cause Centos 5 only support an old release of Ruby how is not compatible with Graylog2 and other dependencies.