A blog to share security, networking and cloud related technology information as @vCloudernBeer picked up on his search for his destiny in the cloud. (LinkedIn: https://www.linkedin.com/in/chowanthony)

Tuesday, November 11, 2014

OpenStack Series: Part 11 Ceilometer – Monitoring and Metering Service

Project Goal/Objective
The Ceilometer project was started in 2012 with one simple goal in mind:to provide an infrastructure
to collect any information needed regarding OpenStack projects. It was designed
so that rating engines could use this single source to transform events into
billable items.

Note: Billing involve 3 steps metering, rating and billing. Ceilometer only deals with metering. Application will have to be build on top of Ceilometer to perform the complete billing process.

As the project become mature a secondary goal is added: become a standard way to collect metric, regardless of the
purpose of the collection

There is also opportunity to work with Heat to provide a tool to watch for variations in key values in order to trigger various reactions.
To summarize there are 3 tasks that OpenStack Ceilometer do:

In Ceilometer data/meter is transformed and published to the receiver.

A single piece or a single set of data/meter can be used by multiple entities. These entities that uses the data/meter has different requirement as to how the data is collected. From the OpenStack Documentation it outlines 2 aspect of data collection that can pose conflict to the data depending on the requester of the data:

Frequency of publication. Typically a meter that you publish for
billing need will need to be updated every 30 min while the same meter needed
for performance tuning may be needed every 10 seconds.

Transport. In the case of data intended for a monitoring system,
losing an update or not ensuring security (non-repudiability) of a message is
not really a problem, while the same meter will need both security and
guaranteed delivery in the case of data intended for rating and billing
systems.

To solve this problem, come the multi-publisher allowing the same technical meter to be published
multiple times to multiple destinations, each potentially using a different
transport and frequency of publication. Three
transports have been implemented so far:

notifier - a notification based
publisher which pushes samples to a message queue.

rpc - he original and
relatively secure RPC based publisher.

udp - which publishes samples using
UDP packets.

This diagram show a single sample is sent to 2 of the transports - RPC and UDP which in turns publish the same data to 2 different target:

Alarming
This component is introduced in the Havana release. It allows user to set alarms based on threshold evaluation for a
collection of samples. An alarm can be set on a single meter, or on a
combination.

There are 2 actions that can be associated with an alarm:

HTTP callback: you provide a URL to be called whenever the alarm has
been set off. The payload of the request contains all the details of why the
alarm was triggered.

The biggest use case for this is to integrate this with OpenStack Heat to provide autoscaling of resources in the OpenStack Infrastructure. For a more detailed description of how this work you can go to this eNorance blog.