How to Configure the Logstash Metrics filter

This short guide will look at the Logstash Metrics filter. It often happens that you’re not interested in actual events, but just the number of times an event occurs in a specific period of time. Logstash supports counting events and measuring their velocity rates through the metrics filter.

Quick Info

The short version

The metrics filter can operate in one of two modes: Meter or timer. In meter mode it will count the number of events, and emit a new event containing the total number of events and the short, mid and long term rates of events at regular intervals. In timer mode it will emit a new event containing the count, short, mid and long term rates and various statistics about the number of events that passed through it at set intervals.

The meter filter will create a new event every 5 seconds, giving you the following stats of events up to this point:

log_events.count – The total number of events up to now

log_events.rate_1m – The average number of events per minute

log_events.rate_5m – The average number of events in 5 minutes

log_events.rate_15m – The average number of events in 15 minutes

You’ll notice that the value of the meter option was used to name the event properties. This is useful to differentiate between different metrics.

The timer filter will create a new event every 5 seconds, giving you the same stats of the event as the meter filter above, as well as the following stats on the request_time property of the event:

request_time.min – The smallest request_time value the filter has seen so far.

request_time.max – The largest request_time value the filter has seen so far.

request_time.stddev – The standard deviation of the value the filter has seen so far.

request_time.mean – The mean of the values the filter has seen so far.

request_time.p1 – The 1th percentile for the measured values.

request_time.p5 – The 5th percentile for the measured values.

request_time.p10 – The 10th percentile for the measured values.

request_time.p90 – The 90th percentile for the measured values.

request_time.p95 – The 95th percentile for the measured values.

request_time.p99 – The 99th percentile for the measured values.

request_time.p100 – The 100th percentile for the measured values.

Once again the value passed as the key of the hash is used to construct the event properties.

The longer version

The metrics filter has a couple of options with which you can tweak it’s behaviour to suit your own needs. By default Logstash will keep on counting events and report on all the events it has received so far until it is stopped. The first example shows how you can use the clear_interval setting to reset the count ever 60 minutes: