How do Sensu filters work?

Sensu filters are applied when event handlers are configured to use one or
more filters. Prior to executing a handler, the Sensu backend will apply any
filters configured for the handler to the event data. If the event is not
removed by the filter(s), the handler will be executed. The
filter analysis flow performs these steps:

When the Sensu backend is processing an event, it will check for the definition
of a handler (or handlers). Prior to executing each handler, the Sensu
server will first apply any configured filters for the handler.

If multiple filters are configured for a handler, they are executed
sequentially.

Filter expressions are compared with event data.

Filters can be inclusive (only matching events are handled) or exclusive
(matching events are not handled).

As soon as a filter removes an event, no further
analysis is performed and the event handler will not be executed.

NOTE: Filters specified in a handler set definition have no effect. Filters must
be specified in individual handler definitions.

Inclusive and exclusive filtering

Filters can be inclusive"action": "allow" (replaces "negate": false in
Sensu 1) or exclusive"action": "deny" (replaces "negate": true in Sensu
1). Configuring a handler to use multiple inclusive filters is the equivalent
of using an AND query operator (only handle events if they match
inclusive filter x AND y AND z). Configuring a handler to use multiple
exclusive filters is the equivalent of using an OR operator (only
handle events if they don’t match x OR y OR z).

Inclusive filtering: by setting the filter definition attribute "action":
"allow", only events that match the defined filter expressions are handled.

Exclusive filtering: by setting the filter definition attribute "action":
"deny", events are only handled if they do not match the defined filter
expressions.

Filter expression comparison

Filter expressions are compared directly with their event data counterparts. For
inclusive filter definitions (like "action": "allow"), matching expressions
will result in the filter returning a true value; for exclusive filter
definitions (like "action": "deny"), matching expressions will result in the
filter returning a false value, and the event will not pass through the
filter. Filters that return a true value will continue to be processed via
additional filters (if defined), mutators (if defined), and handlers.

Filter expression evaluation

When more complex conditional logic is needed than direct filter expression
comparison, Sensu filters provide support for expression evaluation using
Otto. Otto is an ECMAScript 5 (JavaScript) VM,
and evaluates javascript expressions that are provided in the filter.
There are some caveats to using Otto; most notably, the regular expressions
specified in ECMAScript 5 do not all work. See the Otto README for more details.

Filter assets

Sensu filters can have assets that are included in their execution context.
When valid assets are associated with a filter, Sensu evaluates any
files it finds that have a “.js” extension before executing a filter. The
result of evaluating the scripts is cached for a given asset set, for the
sake of performance. For an example of how to implement a filter as an asset, see the guide on reducing alert fatigue.

Built-in filters

Built-in filter: only incidents

The incidents filter is included in every installation of the Sensu backend.
You can use the incidents filter to allow only high priority events through a Sensu pipeline.
For example, you can use the incidents filter to reduce noise when sending notifications to Slack.
When applied to a handler, the incidents filter allows only warning ("status": 1), critical ("status": 2), and resolution events to be processed.

To use the incidents filter, include the is_incident filter in the handler configuration filters array:

When applied to a handler configuration, the not_silenced filter silences events that include the silenced attribute. The handler in the example above uses both the silencing and incidents filters, preventing low priority and silenced events from being sent to Slack.

Built-in filter: has metrics

The metrics filter is included in every installation of the Sensu backend.
When applied to a handler, the metrics filter allows only events containing Sensu metrics to be processed.
You can use the metrics filter to prevent handlers that require metrics from failing in case of an error in metric collection.

To use the metrics filter, include the has_metrics filter in the handler configuration filters array:

Filter specification

Top-level attributes

Required for filter definitions in wrapped-json or yaml format for use with sensuctl create.

type

String

example

"type": "EventFilter"

api_version

description

Top-level attribute specifying the Sensu API group and version. For filters in this version of Sensu, this attribute should always be core/v2.

required

Required for filter definitions in wrapped-json or yaml format for use with sensuctl create.

type

String

example

"api_version": "core/v2"

metadata

description

Top-level collection of metadata about the filter, including the name and namespace as well as custom labels and annotations. The metadata map is always at the top level of the filter definition. This means that in wrapped-json and yaml formats, the metadata scope occurs outside the spec scope. See the metadata attributes reference for details.

required

Required for filter definitions in wrapped-json or yaml format for use with sensuctl create.

Custom attributes you can use to create meaningful collections that can be selected with API filtering and sensuctl filtering. Overusing labels can impact Sensu’s internal performance, so we recommend moving complex, non-identifying metadata to annotations.

required

false

type

Map of key-value pairs. Keys can contain only letters, numbers, and underscores, but must start with a letter. Values can be any valid UTF-8 string.

default

null

example

"labels": {"environment": "development",
"region": "us-west-2"}

annotations

description

Non-identifying metadata that’s meaningful to people or external tools interacting with Sensu.

Handling non-production events

The following filter discards events with a custom entity label "environment": "production", allowing only events without an environment label or events with environment set to something other than production to be handled.
Note that action is deny, making this an exclusive filter; if evaluation
returns false, the event is handled.

Handling repeated events

The following example filter definition, entitled filter_interval_60_hourly,
will match event data with a check interval of 60 seconds, and an
occurrences value of 1 (the first occurrence) -OR- any occurrences
value that is evenly divisible by 60 via a modulo
operator calculation
(calculating the remainder after dividing occurrences by 60).

Handling events during office hours only

This filter evaluates the event timestamp to determine if the event occurred
between 9 AM and 5 PM UTC on a weekday. Remember that action is equal to
allow, so this is an inclusive filter. If evaluation returns false, the event
will not be handled.

Using JavaScript libraries with Sensu filters

You can include JavaScript libraries in their filter execution context with
assets. For instance, assuming you’ve packaged underscore.js into a Sensu
asset, you could then use functions from the underscore library for filter
expressions.

About Sensu

The Sensu monitoring event pipeline empowers businesses to automate their monitoring workflows and gain deep visibility into their multi-cloud infrastructure, from Kubernetes to bare metal. Companies like Sony, Box.com, and Activision rely on Sensu to help deliver value faster, at scale.