Content

Bitnami ELK Installer

NOTE: Before running the commands shown on this page, you should load the Bitnami stack environment by executing the installdir/use_APPNAME script (Linux and Mac OS X) or by clicking the shortcut in the Start Menu under "Start -> Bitnami APPNAME Stack -> Application console" (Windows). Learn more.

NOTE: When running the commands shown on this page, replace the installdir placeholder with the full installation directory for your Bitnami stack.

Click the "Add" button, select the previous visualization and save the dashboard.

What is the default configuration?

ELK default configuration

Elasticsearch configuration file

The main configuration file for Elasticsearch is installdir/elasticsearch/config/elasticsearch.yml.

Elasticsearch ports

By default, Elasticsearch will use port 9200 for requests and port 9300 for communication between nodes within the cluster. If these ports are in use when the server starts, it will attempt to use the next available port, such as 9201 or 9301.

Set custom ports using the configuration file, together with details such as the cluster name (elasticsearch by default), node name, address binding and discovery settings. All these settings are needed to add more nodes to your Elasticsearch cluster.

Elasticsearch log file

The ELK log file is created at installdir/elasticsearch/logs/elasticsearch.log.

Logstash default configuration

Logstash configuration file

The main configuration file for Logstash is installdir/logstash/conf/logstash.conf.

Logstash port

By default, Logstash will use port 9600. If this port is in use when the server starts, it will attempt to use the next available port, such as 9601.

Logstash log file

The Logstash log file is created at installdir/logstash/logs/logstash.log.

Kibana default configuration

Kibana configuration file

The main configuration file for Kibana is installdir/kibana/config/kibana.yml.

Kibana ports

By default, Kibana will use port 5601. If this port is in use when the server starts, it will attempt to use the next available port, such as 5602.

You can set a custom port using the configuration file, together with details such as the Elasticsearch URL (http://127.0.0.1:9200 by default), Kibana index, default application to load or verbosity level.

Kibana log file

The Kibana log file is created at installdir/kibana/logs/kibana.log.

How to change the Elasticsearch password?

Follow the steps below to change the Elasticsearch password:

Execute the following command. You will be prompted to enter a new password for the user user.

How to connect remotely?

How to connect remotely to Elasticsearch?

IMPORTANT: Bitnami Native Installers do not modify the firewall configuration of your computer, therefore the ELK ports could be open which is a significant security risk. You are strongly advised to close the ELK ports (refer to the FAQ for more information on this).

To access the ELK server from another computer or application, make the following changes to the node's installdir/elasticsearch/config/elasticsearch.yml file:

network.host: Specify the hostname or IP address where the server will be accesible. Set it to 0.0.0.0 to listen on every interface.

network.publish_host: Specify the host name that the node publishes to other nodes for communication.

How to connect remotely to Logstash using SSL certificates?

It is strongly recommended to create an SSL certificate and key pair in order to verify the identity of ELK Server. In this example, we are going to use Filebeat to ship logs from our client servers to our ELK server:

Add the ELK Server's private IP address to the subjectAltName (SAN) field of the SSL certificate on the ELK server. To do so, open the OpenSSL configuration file (installdir/common/openssl/openssl.cnf), find the [ v3_ca ] section in the file, and add this line under it (substitute in the ELK server's private IP address for the IP_ADDRESS placeholder):

subjectAltName = IP: IP_ADDRESS

Generate the SSL certificate and private key in the appropriate locations (e.g. installdir/logstash/ssl/), with the following commands:

How to start or stop the services?

Linux

Bitnami native installers include a graphical tool to manage services. This tool is named manager-linux-x64.run on Linux and is located in the installation directory. To use this tool, double-click the file and then use the graphical interface to start, stop or restart services. Server log messages can be checked in the "Server Events" tab.

The native installer also includes a command-line script to start, stop and restart applications, named ctlscript.sh. This script can be found in the installation directory and accepts the options start, stop, restart, and status. To use it, log in to the server console and execute it following the examples below:

Call it without any service names to start all services:

$ sudo installdir/ctlscript.sh start

Use it to restart a specific service only by passing the service name as argument - for example, mysql, postgresql or apache:

The list of available services varies depending on the required components for each application.

Mac OS X

Bitnami native installers include a graphical tool to manage services. This tool is named manager-osx on Mac OS X and is located in the installation directory. To use this tool, double-click the file and then use the graphical interface to start, stop or restart services. Server log messages can be checked in the "Server Events" tab.

The native installer also includes a command-line script to start, stop and restart applications, named ctlscript.sh. This script can be found in the installation directory and accepts the options start, stop, restart, and status. To use it, log in to the server console and execute it following the examples below:

Call it without any service names to start all services:

$ sudo installdir/ctlscript.sh start

Use it to restart a specific service only by passing the service name as argument - for example, mysql or apache:

The list of available services varies depending on the required components for each application.

NOTE: If you are using the stack manager for Mac OS X-VM, please check the following blog post to learn how to manage services from its graphical tool.

Windows

Bitnami native installers include a graphical tool to manage services. This tool is named manager-windows.exe on Windows and is located in the installation directory. To use this tool, double-click the file and then use the graphical interface to start, stop or restart services. Server log messages can be checked in the "Server Events" tab.

The Windows native installer creates shortcuts to start and stop services created in the Start Menu, under "Programs -> Bitnami APPNAME Stack -> Bitnami Service". Servers can also be managed from the Windows "Services" control panel. Services are named using the format APPNAMESERVICENAME, where APPNAME is a placeholder for the application name and SERVICENAME is a placeholder for the service name. For example, the native installer for the Bitnami WordPress Stack installs services named wordpressApache and wordpressMySQL.

How to install a plugin on Logstash?

Logstash supports input, filter, codec and output plugins. These are available as self-contained gems (RubyGems.org). You can install, uninstall and upgrade plugins using the Command Line Interface (CLI) invocations described below:

Install a plugin:

$ cd installdir/logstash
$ bin/logstash-plugin install PLUGIN

Update a plugin:

$ bin/logstash-plugin update PLUGIN

List all installed plugins:

$ bin/logstash-plugin list

Uninstall a plugin (for Logstash <= 2.4 versions):

$ bin/logstash-plugin uninstall PLUGIN

How to install a plugin on Kibana?

Add-on functionality for Kibana is implemented with plug-in modules.

Install a plugin:

$ cd installdir/kibana
$ bin/kibana-plugin install ORG/PLUGIN/VERSION

List all installed plugins:

$ bin/kibana-plugin list

Remove a plugin:

$ bin/kibana-plugin remove PLUGIN

You can also install a plugin manually by moving the plugin file to the plugins directory and unpacking the plugin files into a new directory.

How to create a full backup of Elasticsearch data?

Backup

Elasticsearch provides a snapshot function that you can use to back up your data. Follow these steps:

Register a repository where the snapshot will be stored. This may be a local directory or cloud storage (which requires additional plugins). In this example, we will use a local repository, which can be initialized via the Elasticsearch REST API with the following commands:

In this example, my_backup is the name of the repository created previously and snapshot_1 is the name for the backup. The wait_for_completion option will block the command line until the snapshot is complete. To create the snapshot in the background, simply omit this option, as shown below:

$ curl -XPUT 'localhost:9200/_snapshot/my_backup/snapshot_1'

Restore

To restore a backup over existing data, follow these steps:

Close the specific indices that will be overwritten with this command:

$ curl -XPOST 'localhost:9200/my_index/_close'

Optionally, close all indices:

$ curl -XPOST 'localhost:9200/_all/_close'

Restore the backup with the following command. This command will also reopen the indices closed before.

How to upgrade ELK?

NOTE: It's highly recommended to perform a backup before any upgrade.

Upgrade Elasticsearch

Since version 0.90.7, Elasticsearch supports rolling upgrades. As a result, it's not necessary to stop the entire cluster during the upgrade process. Instead, it is possible to upgrade one node at a time and keep the rest of the cluster operating normally.

Upgrade Kibana

Extract to a new directory (not overwriting the current installation) - for example, /tmp/new_kibana.

Take note of the Kibana plugins that are already installed:

$ kibana/bin/kibana-plugin list

Backup old files:

$ cd installdir
$ sudo cp kibana old_kibana

Copy files from new installation directory:

$ sudo cp -r /tmp/new_kibana/* kibana/

Recover the kibana.yml file:

$ cp old_kibana/config/kibana.yml kibana/config/kibana.yml

Start the service again:

$ sudo installdir/ctlscript.sh start kibana

How to enable HTTPS support with SSL certificates?

NOTE: The steps below assume that you are using a custom domain name and that you have already configured the custom domain name to point to your cloud server.

Bitnami images come with SSL support already pre-configured and with a dummy certificate in place. Although this dummy certificate is fine for testing and development purposes, you will usually want to use a valid SSL certificate for production use. You can either generate this on your own (explained here) or you can purchase one from a commercial certificate authority.

Once you obtain the certificate and certificate key files, you will need to update your server to use them. Follow these steps to activate SSL support:

Use the table below to identify the correct locations for your certificate and configuration files.

NOTE: If you use different names for your certificate and key files, you should reconfigure the SSLCertificateFile and SSLCertificateKeyFile directives in the corresponding Apache configuration file to reflect the correct file names.

If your certificate authority has also provided you with a PEM-encoded Certificate Authority (CA) bundle, you must copy it to the correct location in the previous table. Then, modify the Apache configuration file to include the following line below the SSLCertificateKeyFile directive. Choose the correct directive based on your scenario and Apache version:

Variable

Value

Apache configuration file

installdir/apache2/conf/bitnami/bitnami.conf

Directive to include (Apache v2.4.8+)

SSLCACertificateFile "installdir/apache2/conf/server-ca.crt"

Directive to include (Apache < v2.4.8)

SSLCertificateChainFile "installdir/apache2/conf/server-ca.crt"

NOTE: If you use a different name for your CA certificate bundle, you should reconfigure the SSLCertificateChainFile or SSLCACertificateFile directives in the corresponding Apache configuration file to reflect the correct file name.

Once you have copied all the server certificate files, you may make them readable by the root user only with the following commands:

How to create an SSL certificate?

OpenSSL is required to create an SSL certificate. A certificate request can then be sent to a certificate authority (CA) to get it signed into a certificate, or if you have your own certificate authority, you may sign it yourself, or you can use a self-signed certificate (because you just want a test certificate or because you are setting up your own CA).

Follow the steps below for your platform.

Linux and Mac OS X

NOTE: OpenSSL will typically already be installed on Linux and Mac OS X. If not installed, install it manually using your operating system's package manager.

Note that if you use this encrypted key in the Apache configuration file, it will be necessary to enter the password manually every time Apache starts. Regenerate the key without password protection from this file as follows:

Windows

Set the OPENSSL_CONF environment variable to the location of your OpenSSL configuration file. Typically, this file is located in the bin/ subdirectory of your OpenSSL installation directory. Replace the OPENSSL-DIRECTORY placeholder in the command below with the correct location.

$ set OPENSSL_CONF=C:\OPENSSL-DIRECTORY\bin\openssl.cfg

Change to the bin/ sub-directory of the OpenSSL installation directory. Replace the OPENSSL-DIRECTORY placeholder in the command below with the correct location.

Note that if you use this encrypted key in the Apache configuration file, it will be necessary to enter the password manually every time Apache starts. Regenerate the key without password protection from this file as follows:

How to debug Apache errors?

Once Apache starts, it will create two log files at installdir/apache2/logs/access_log and installdir/apache2/logs/error_log respectively.

The access_log file is used to track client requests. When a client requests a document from the server, Apache records several parameters associated with the request in this file, such as: the IP address of the client, the document requested, the HTTP status code, and the current time.

The error_log file is used to record important events. This file includes error messages, startup messages, and any other significant events in the life cycle of the server. This is the first place to look when you run into a problem when using Apache.

If no error is found, you will see a message similar to:

Syntax OK

Updating the IP address or hostname

ELK requires updating the IP address/domain name if the machine IP address/domain name changes. The bnconfig tool also has an option which updates the IP address, called --machine_hostname (use --help to check if that option is available for your application). Note that this tool changes the URL to http://NEW_DOMAIN/elk.

$ sudo installdir/apps/elk/bnconfig --machine_hostname NEW_DOMAIN

If you have configured your machine to use a static domain name or IP address, you should rename or remove the installdir/apps/elk/bnconfig file.

NOTE: Be sure that your domain is propagated. Otherwise, this will not work. You can verify the new DNS record by using the Global DNS Propagation Checker and entering your domain name into the search field.

You can also change your hostname by modifying it in your hosts file. Enter the new hostname using your preferred editor.

$ sudo nano /etc/hosts

Add a new line with the IP address and the new hostname. Here's an example. Remember to replace the IP-ADDRESS and DOMAIN placeholders with the correct IP address and domain name.

IP-ADDRESS DOMAIN

Troubleshooting

Elasticsearch has strict kernel requirements. You may find the issues below when starting Elasticsearch service:

How to make the Kibana dashboard public?

NOTE: For security reasons, we do not recommend disabling authentication.

By default, you will be prompted for a username and password every time you access the Kibana dashboard. This can create problems if, for example, you wish to embed Kibana data in other pages. To disable the authentication prompt, follow these steps:

Edit the Apache configuration file at installdir/elasticsearch/apache-conf/elasticsearch.conf and remove the LocationMatch section.

Restart Apache by running the command below:

$ sudo installdir/ctlscript.sh restart apache

How to install elasticsearch-head?

Elasticsearch-head is a Web front-end for an Elasticsearch cluster. For Elasticsearch 5.x, site plugins are not supported, so it needs to run as a standalone server. Follow these steps:

Install Node.js and npm. For example, the commands below will install them on Debian:

Update the installdir/elasticsearch/config/elasticsearch.yml file and enable CORS by setting http.cors.enabled to true:

http.cors.enabled: true

In the same file, set the http.cors.allow-origin variable to the domains that are allowed to send cross-origin requests. If you prepend and append a "/" to the value, this will be treated as a regular expression. For example:

http.cors.allow-origin: /https?:\/\/localhost(:[0-9]+)?/

NOTE: You can set the value of http.cors.allow-origin to "*" to allow CORS requests from anywhere if you wish. However, this is not recommended as it is a security risk.

Browse to http://localhost/elasticsearch-head/?base_uri=http://localhost/elasticsearch and insert your Elasticsearch credentials. You should see something like the screenshot below:

Which components are installed with the Bitnami ELK Stack?

The Bitnami ELK Stack ships the components listed below. If you want to know which specific version of each component is bundled in the stack you are downloading, check the README.txt file on the download page or in the stack installation directory. You can also find more information about each component using the links below.