This tutorial assumes you have Elasticsearch and Kibana installed and
accessible from Filebeat (see the getting started
section). It also assumes that the Ingest Node GeoIP and User Agent plugins are
installed. These plugins are required to capture the geographical location and
browser information used by some of the visualizations available in the sample
dashboards. You can install these plugins by running the following commands in the Elasticsearch home path:

If you are using an Elastic Cloud instance, you can
enable the two plugins from the configuration page.

This also assumes you have Nginx installed and writing logs in the default
location and format. If you want to monitor another service for which a module
exists, adjust the commands in the tutorial accordingly.

You can start Filebeat with the following command:

./filebeat -e -modules=nginx -setup

The -e flag tells Filebeat to output its logs to standard error, instead of
syslog.

The -modules=nginx flag loads the Nginx module.

The -setup flag tells Filebeat to load the associated sample Kibana
dashboards. This setup phase, in which the dashboards are loaded, doesn’t have
to be executed each time, and because it’s a relatively heavy operation, we
recommend executing it only once after installing or upgrading Filebeat. That
is why, the next commands from this tutorial are omitting the -setup flag.

Visiting the Kibana web interface now, open the Nginx dashboard and you should
already see your logs parsed and visualized in several widgets.

You can also start multiple modules at once:

./filebeat -e -modules=nginx,mysql,system

Because Filebeat modules are currently in Beta, the default Filebeat
configuration may interfere with the Filebeat system module configuration. If
you plan to run the system module, edit the Filebeat configuration file,
filebeat.yml, and comment out the following lines:

#- input_type: log
#paths:
#- /var/log/*.log

For rpm and deb, you’ll find the configuration file at
/etc/filebeat/filebeat.yml. For mac and win, look in the archive that you
extracted when you installed Filebeat.

While enabling the modules from the CLI file is handy for getting started and
for testing, you will probably want to use the configuration file for the
production setup. The equivalent of the above in the configuration file is:

Each module and fileset has a set of "variables" which allow adjusting their
behaviour. To see the available variables, you can consult the
filebeat.full.yml file. For example, all filesets allow setting a custom
paths value, which is a list of Globs where the log files are searched.

These variables have default values, sometimes depending on the operating
system. You can override them either from the CLI via the -M flag, or from
the configuration file.

In the case of Nginx, for example, you can use the following if the access
files are in a custom location:

The Nginx access fileset also has a pipeline variable which allows
selecting which of the available Ingest Node pipelines is used for parsing. At
the moment, two such pipelines are available, one that requires the two ingest
plugins (ingest-geoip and ingest-user-agent) and one that doesn’t. If you
cannot install the plugins, you can use the following:

Behind the scenes, each module starts a Filebeat prospector. For advanced
users, it’s possible to add or overwrite any of the prospector settings. For
example, enabling close_eof can be done like this: