Installation

This function can be automatically deployed using a Serverless Application Model (SAM) in CloudFormation. Use the links below based on the specified region to walk through the CloudFormation deployment model.

You must supply parameters for your cluster name, endpoint address and port, master username and the encrypted password, and the aggregation interval to be used by the monitoring scripts (default 1 hour).

The SAM stack will create:

An IAM Role called LambdaRedshiftMonitoringRole

This IAM Role will have a single linked IAM Policy called LambdaRedshiftMonitoringPolicy that can:

Decrypt the KMS Key used to encrypt the cluster password (kms::Decrypt)

Emit CloudWatch metrics (cloudwatch::PutMetricData)

Region

VPC Template

Non-VPC Template

ap-northeast-1

ap-northeast-2

ap-south-1

ap-southeast-1

ap-southeast-2

ca-central-1

eu-central-1

eu-west-1

eu-west-2

eu-west-3

sa-east-1

us-east-1

us-east-2

us-west-1

us-west-2

If you wish to deploy manually, you can use the prebuilt zip in the dist folder, or you can build it yourself. We've included a build script for bash shell that will create a zip file which you can upload into AWS Lambda.

Configuring with Events (Best)

This option allows you to send the configuration as part of the Scheduled Event, which then means you can support multiple clusters from a single Lambda function. This option will override any Environment variables that you've configured. An example event looks like:

The old environment variable names are provided for backward compatibility, but you can use environment variables with the above names, and it will use those instead.

Manual Deployment Instructions

If you are rebuilding the function, download and install dependencies

pip install -r requirements.txt -t .

Assemble and compress the Lambda function package:

./build.sh

If you are including any user defined query extensions, then build with:

./build.sh --include-user-queries

Please note the labelled version in Github does not include any user queries

Create a lambda function, some of the parameters of the function are:

Runtime: Python 2.7

Upload the zip file generated

Handler: lambda_function.lambda_handler

Role: Use the role created

Memory: 256MB

Timeout: 5 minutes

VPC: Use the same VPC as the Redshift cluster. You're going to need at least two private subnets with access to the Redshift cluster in its Security Group. You should have a NAT Gateway to give access to Internet to those subnets routing tables. You cannot use public subnets. You can read more information here AWS blog

Add an Event Source to the Lambda function with a Scheduled Event, running with the same frequency you configured in the Lambda function.

Confirming Successful Execution

After a period of time, you can check your CloudWatch metrics, and create alarms. You can also create a Dashboard with all the graphs and have a view of your database as this one:

Extensions

The published CloudWatch metrics are all configured in a JSON file called monitoring-queries.json. These are queries that have been built by the AWS Redshift database engineering and support teams and which provide detailed metrics about the operation of your cluster.

If you would like to create your own queries to be instrumented via AWS CloudWatch, such as user 'canary' queries which help you to see the performance of your cluster over time, these can be added into the user-queries.json file. The file is a JSON array, with each query having the following structure:

The last attribute type is probably the most important. If you use value, then the value from your query will be exported to CloudWatch with the indicated unit and Metric Name. However, if you use interval, then the runtime of your query will be instrumented as elapsed milliseconds, giving you the ability to create the desired 'canary' query.

Copyright 2016-2017 Amazon.com, Inc. or its affiliates. All Rights Reserved.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.