vPoller -- VMware vSphere Distributed Poller written in Python

vPoller is a distributed system written in Python for discoverying and
polling of vSphere Objects properties.

vPoller uses the vSphere
API in order to
perform discovery and polling of Objects from a vSphere host
(e.g. VMware ESXi or VMware vCenter server instance).

The vPoller distributed system consists of a number of components,
each performing a different task:

vpoller-proxy - A ZeroMQ proxy which load-balances client requests to a pool of workers

vpoller-worker - A vSphere Worker, which does the actual polling and discovering from a vSphere host

vpoller-client - A client program used for sending and receiving requests to a vSphere Worker

On the diagram below you can view the principal work of the vPoller
distributed system:

The distributed nature of vPoller is provided by
ZeroMQ, which also load balances client requests
to a pool of vSphere Workers.

vPoller was designed to be easy for integration into systems which
require access to vSphere Objects properties, but do not have native
support for it.

Possible scenarios where vPoller could be used is to integrate it
into monitoring systems as part of the polling process in order to
provide monitoring of your VMware vSphere environment. It could also
be used in applications for collecting statistics and other metrics
from your VMware vSphere environment.

vPoller can also be described as a VMware vSphere API-Proxy, because
it translates user requests to vSphere API requests thus allowing the
user to use the API without having the need to know how the vSphere
API works internally.

vPoller has been tested with vSphere 5.x and with very limited testing
on vSphere 4.x

In this article we will see how to install, configure and manage the
vPoller components.

For more and up-to-date information about vPoller you are advised to
also check the vPoller documentation on
Github, which contains more
comprehensive information about the components and their purpose.

Here we will focus on the actual usage of the vPoller distributed
system in order to perform discovery and polling of vSphere Object
properties.

Installation

First, make sure that you have all the requirements listed above
installed before starting with the installation of vPoller.

In order to install vPoller simply execute the command below:

$ sudo python setup.py install

We need to create a few directories as well, which are used by the
vPoller components to store their log files and lock files as well, so
make sure you create them first:

$ sudo mkdir /var/log/vpoller /var/run/vpoller

Now, go to the next section where we will see how to configure the
vPoller components.

Configuration

First, let’s configure the vPoller Proxy, which is component that
takes care of dispatching the client requests to our vPoller Workers
and does the automatic load-balancing using
ZeroMQ.

The default configuration file of the vpoller-proxy resides in
/etc/vpoller/vpoller-proxy.conf, although you can specify a
different config file from the command-line as well.

Below is an example configuration file used by the vpoller-proxy:

[Default]frontend=tcp://*:10123backend=tcp://*:10124mgmt=tcp://*:9999

Now, let’s configure the vPoller Workers. The vPoller Worker is what
does the actual polling and discovering of vSphere objects from a
vSphere host.

Internally it runs the vSphere Agents, which take care of connecting
to a vSphere host, initiating the user’s API requests and send back
the result to the client.

A vPoller Worker can connect to any number of vSphere hosts, where
each vSphere connection is handled by a separate vSphere Agent object.

In this example we will be running a vPoller Worker on two nodes of
our infrastructure in order to provide redundancy and
load-balancing. Each vPoller Worker will be connected to a single
VMware vCenter server instance from where we will be getting our
results.

The default configuration file of the vpoller-worker resides in
/etc/vpoller/vpoller-worker.conf, although you can specify a
different config file from the command-line as well.

Now, it is time to configure the vSphere Agents of our vPoller
Worker. A vSphere Agent is what takes care of establishing a
connection to the vSphere host and perform any poll or discovery
operations. For each vSphere host you want to use you need a
configuration file describing the connection details.

Below is an example configuration file
/etc/vpoller/vsphere/my-vc0.conf, which describes the connection
details to our vCenter server.

Discovering ESXi hosts on a vCenter server

Time to test out vPoller. In this example we will see how we can
discover all ESXi hosts, which are already registered to our VMware
vCenter server instance.

We will be using vpoller-client tool in order to send requests to
our vPoller Workers.

For more information about the different command-line options
supported by vpoller-client, execute the command below to get usage
information.

$ vpoller-client --help

Our vPoller Proxy endpoint is tcp://localhost:10123.

Considering that the vPoller Proxy endpoint is at
tcp://localhost:10123 in order to discover the ESXi hosts on vCenter
vc01-test.example.org you would execute a similar command:

On the screenshot below you can see how we send a request to our
vPoller Proxy, which in turn is dispatched to the vPoller Workers
connected to the Proxy. Once the request is processed by our vPoller
Workers result is sent back to the client in JSON format.

The returned result shows that we have only a single ESXi host
registered to our vCenter server instance.

In the next examples we will use the information that we’ve discovered
already in order to perform polling of Object properties.

Discovering Datastores on a vCenter server

Now, let’s see how we can discover all datastores on our vCenter
server instance.

On the screenshot below you can see the request we send to our
vPoller Proxy and the result returned to the client from the
vPoller Workers.

The returned result shows us all datastores that we have on our
vCenter server instance.

In the next examples we will use the information that we’ve discovered
already in order to perform polling of Object properties.

Polling vSphere Object Properties

Let’s see now how we can perform polling of vSphere Object
properties. By using the information we’ve got from discoverying the
ESXi hosts and datastores from our vCenter we will now send a request
for different properties of our vSphere Objects.

Example request to get the power state of an ESXi host. The property
we want to retrieve is runtime.powerState.

What amout of memory do we have on our ESXi host? The property we want
to retrieve is hardware.memorySize.

Let’s see what was the time our ESXi hosts booted. The property we
want to retrieve is runtime.bootTime.

Okay, let’s poll some properties now about datastores. Using the
discovery information we got previously we will now see how to get
various information about our datastores.

What is the capacity of our datastore? The property we want to
retrieve is summary.capacity.

What is the name of our datastore? The property we want ot retrieve is
info.name.

Let’s see the amount of free space we have on our datastore. The
property we want to retrieve is info.freeSpace.

vPoller Helpers

The vPoller Helpers were implemented in order to provide an easy way
for connecting your applications to vPoller.

The result messages returned by a vPoller Worker are always in JSON
format.

This could be okay for most applications, which require to process a
result message, but in some cases you might want to receive the result
in different formats.

By using the vPoller Helpers you are able to convert any result
messages to a format which your application can understand and
use. This is useful for presenting the information in a way which you
can feed it to your application.