Background

This ZenPack provides support for monitoring Amazon Web Services (AWS).
Monitoring for the following EC2, VPC and S3 entities is provided through a combination of
the AWS EC2 and CloudWatch APIs.

Note: This ZenPack supersedes the older ZenAWS (ZenPacks.zenoss.ZenAWS) ZenPack that was installed by default on versions of Zenoss prior to 4.2.4. Please remove ZenAWS before installing this ZenPack. This will remove the /EC2 device class and the EC2Manager device within. After installing this ZenPack, you will be able to add a new EC2 Account with much greater functionality.

Optional service impact with addition of Zenoss Service Dynamics product.

Discovery

The following entities will be automatically discovered through an account
name, access key and secret key you provide. The attributes, tags and
collections will be updated on Zenoss' normal remodeling interval which
defaults to every 12 hours.

Monitoring

The following metrics will be collected every 5 minutes by default. Any other
CloudWatch metrics can also be collected by adding them to the appropriate
monitoring template. The Average statistic is collected, and the graphed
value is per second for anything that resembles a rate.

The Amazon CloudWatch datasource type also allows for the collection of any other CloudWatch metric.

Besides CloudWatch metrics, the following metrics will also be collected every 5 minutes by default.

Subnets

Metrics: Available IP Adresses count

S3 Buckets

Metrics: Keys count, Total Size

Monitoring large cloud may require to contact AWS support with request to increase CloudWatch API requests limit.
Appropriate event will be created in Zenoss in case limit for CloudWatch requests has been exceeded.

CloudWatch datasources utilize multithreading for better performance. It is possible to increase speed by setting "twistedthreadpoolsize" value in configuration of "zenpython" daemon. Please note that setting higher value will result also in bigger memory usage.

Soft Limits Monitoring

The following resource counts subject to the soft-limits will be collected every 5 minutes and when any of these metrics approaches a soft limit threshold, a Zenoss event will be triggered.

The thresholds are set to the default limit values. If you changed this limit for your account, you should manually change the Max threshold value using the following steps:

Navigate to Monitoring Templates (Advanced panel).

Click EC2Region and find RegionsSoftLimits among Data Sources.

On the Thresholds panel choose the resources count to be changed.

Double click on the resources count and change the value in the Maximum Value field.

Guest Device Discovery

You can optionally configure each monitored AWS account to attempt to discover
and monitor the guest Linux or Windows operating systems running within each
EC2 instance, when specific Tags are present. This requires that your Zenoss system
has the network and server access it needs to monitor the guest operating system.
VPC and non-VPC modes are supported.

The guest operating system devices' life-cycle are managed along with the
instance. For example, the guest operating system device is set to a
decommissioned production state when the EC2 instance is stopped, and the
guest operating system device is deleted when the EC2 instance is destroyed.

Service Impact

When combined with the Zenoss Service Dynamics product, this ZenPack adds
built-in service impact capability for services running on AWS. The following
service impact relationships are automatically added. These will be included
in any services that contain one or more of the explicitly mentioned entities.

Usage

Adding AWS Accounts

Use the following steps to start monitoring EC2 using the Zenoss web interface.

Navigate to the Infrastructure page.

Choose Add EC2 Account from the add device button.

Enter your AWS account name, access key and secret key.

Optionally choose a collector other than the default localhost.

Click Add.

Alternatively you can use zenbatchload to add accounts from the command line.
To do this, you must create a file with contents similar to the following.
Replace all values in angle brackets with your values minus the brackets.
Multiple accounts can be added under the same /Device/AWS/EC2 section.

Configuring filter for modeler plugin

Use zAWSRegionToModel property to narrow components modeled. By default it has empty value,
so all EC2 regions and it's child components will be discovered. Specify EC2 region name,
or multiple names separated by comma in it. This will be used as a filter and may
help with large AWS accounts.

Configuring Guest Device Discovery

Use the following steps to configure instance guest device discovery. Guest
device discovery must be configured individually for each EC2 account.

Navigate to the Configuration Properties panel and in the zAWSDiscover property specify the instances' tags and values (e.g. tag:value;).

Verify that appropriate SSH, SNMP or Windows credentials are configured for the chosen device class(es).

To choose private or public IP address will be used for creating guest device, change the zAWSGuestUsePublicIPs property.

Remodel the EC2 account by choosing Model Device from its menu.

If your instances are VPC instances, and are in a different VPC than the
Zenoss server that's monitoring the EC2 account, you must add a Collector
tag to containing VPC with the value set to the name of the Zenoss collector
to which discovered guest devices should be assigned.

Example:

If zAWSDiscover was filled with the value Test:test; after modeling all the devices with the tag Test:test will be discovered

If zAWSDiscover was filled with the value Test1:test1; Test2:test2 after modeling all the devices with either of the tag will be discovered

Configuring Remote Collector for Guest Devices

You can optionally configure an alternate remote collector for the devices
created from AWS Instances with the following configuration properties:

zAWSGuestCollector

This property allows you to specify the name of the collector all discovered
devices for this AWS device will use.

zAWSResetGuestCollector

Setting this property to false will tell AWS not to change the collector
if you have set it manually.

Configuring Instances Remodeling

You can optionally configure your monitored AWS account, so that the newly
added or recently dropped instances are automatically reflected on Zenoss UI
during monitoring:

Navigate to the Configuration Properties panel.

Enable the zAWSRemodelEnabled property (set it to true, this field is not case sensitive).

PEM file

Use the following steps to specify the PEM file to region for use in auto-discovering instance guest operating systems:

Navigate to the Configuration Properties panel.

Set region name and path to PEM file in the appropriate fields of zAWSRegionPEM property (see zAWSRegionPEM Propery).

Installed Items

Installing this ZenPack will add the following items to your Zenoss system.

Upgrade

The AWS Zenpack of versions 2.0.0 / 2.1.0 can be upgraded. To upgrade the ZenPack, install the
latest version over the existing one. There is no action for the user to migrate the data.
The performance data and events of old ZenPack are retained as per the retain policy settings.

Limitations

In the current version of Zenpack monitoring of large AWS account (ex., > 1000

EC2 instances and volumes) may cause performance issues:

Limit for datapoints processed by zenpython daemon may be exceeded. This will result in gaps in graphs.

Monitoring cycle may not fit into default value of 5 minutes. This will result for some points on graphs to be not aligned by 5 minutes interval.

Having more than one AWS account added into Zenoss may lead to issues described above.

Note: It is possible to reduce number of datapoints collected by disabling monitoring templates you don't need.

Changes

2.4.0

Update boto version shipped with the ZenPack to support new "ap-northeast-2" region.

Improve HTTP errors and warnings.

Added zAWSCloudWatchMaxParallel property to configure number of concurrent cloudwatch calls.

Make the number of retries for cloudwatch calls configurable (zAWSCloudWatchMaxRetries property).

Allow modeler to set it Region explicitlty, and ignore unmodeled buckets.

Added path reporter for EC2Snapshots

2.3.1

Ignore reserved instances with a null id. (ZEN-17556).

Added zAWSRegionToModel property to tell RM what to model (ZEN-17374)

Improved zAWSRemodelEnabled and zAWSResetGuestCollector properties

2.3.0

Add ability for instances into VPC to use public IP address for guest device

Add parallel processing for CloudWatch datasources using multithreading. For large AWS installation it can be boosted by setting bigger value for "twistedthreadpoolsize" setting of PythonCollector.

2.2.2

Add support for Zenoss 5x.

Add ability for user to specify an alternate remote collector for discovered devices.