Posts

It's a bit tricky to figure out how to use a recording rule together with a Prometheus installation using the CoreOS Prometheus Operator. Here's how I got it working.

If you take a look at Documentation/user-guides/Alerting.md, you'll see that Kubernetes ConfigMaps are used to load the Prometheus configuration for each Prometheus (deployed via a CRD) in your cluster. These ConfigMaps are selected via two labels, role: alert-rules and prometheus: <example>.

So if you are using kube-prometheus to deploy the Prometheus Operator to your cluster (it gets a bit complicated with all the layers of abstraction), then if you want to deploy a recording rule directly into the k8s Prometheus instance, you'll want to do this:

Create the file with your recording rules and save it as something like custom-rules.yaml﻿