Using CloudWatch Logs with Container Instances

You can configure your container instances to send log information to CloudWatch Logs.
This enables
you to view different logs from your container instances in one convenient location.
This
topic helps you get started using CloudWatch Logs on your container instances that
were launched with
the Amazon ECS-optimized Amazon Linux AMI.

CloudWatch Logs IAM Policy

Before your container instances can send log data to CloudWatch Logs, you must create
an IAM
policy to allow your container instances to use the CloudWatch Logs APIs, and then
you must attach
that policy to ecsInstanceRole.

To narrow the available policies to attach, for Filter,
type ECS-CloudWatchLogs.

Check the box to the left of the ECS-CloudWatchLogs
policy and choose Attach policy.

Installing the CloudWatch Logs Agent

After you have added the ECS-CloudWatchLogs policy to your
ecsInstanceRole, you can install the CloudWatch Logs agent on your container
instances.

Note

This procedure was written for the Amazon ECS-optimized Amazon Linux AMI, and may
not work on other operating
systems. For information about installing the agent on other operating systems, see
Getting Started with
CloudWatch Logs in the Amazon CloudWatch User Guide.

To install the CloudWatch Logs agent

Run the following command to install the CloudWatch Logs agent.

[ec2-user ~]$ sudo yum install -y awslogs

After you have installed the agent, proceed to the next section to configure the
agent.

Configuring and Starting the CloudWatch Logs Agent

The CloudWatch Logs agent configuration file (/etc/awslogs/awslogs.conf)
describes the log files to send to CloudWatch Logs. The agent configuration file's
[general] section defines common configurations that apply to all log
streams, and you can add individual log stream sections for each file on your container
instances that you want to monitor. For more information, see CloudWatch Logs Agent Reference in
the Amazon CloudWatch User Guide.

The example configuration file below is configured for the Amazon ECS-optimized Amazon
Linux AMI, and it provides
log streams for several common log files:

/var/log/dmesg

The message buffer of the Linux kernel.

/var/log/messages

Global system messages.

/var/log/docker

Docker daemon log messages.

/var/log/ecs/ecs-init.log

Log messages from the ecs-init upstart job.

/var/log/ecs/ecs-agent.log

Log messages from the Amazon ECS container agent.

/var/log/ecs/audit.log

Log messages from the IAM roles for the task credential provider.

You can use the example file below for your Amazon ECS container instances, but you
must
substitute the {cluster} and {container_instance_id} entries
with the cluster name and container instance ID for each container instance so that
the
log streams are grouped by cluster name and separate for each individual container
instance. The procedure that follows the example configuration file has steps to replace
the cluster name and container instance ID placeholders.

By default, the CloudWatch Logs agent sends data to the us-east-1 region. To
send your data to a different region, such as the Region in which your cluster is
located, you can set the Region in the /etc/awslogs/awscli.conf
file.

Open the /etc/awslogs/awscli.conf file with a text
editor.

In the [default] section, replace us-east-1 with the
Region from which to view log data.

Save the file and exit your text editor.

To start the CloudWatch Logs agent

Start the CloudWatch Logs agent with the following command.

[ec2-user ~]$ sudo service awslogs start

Output:

Starting awslogs: [ OK ]

Use the chkconfig command to ensure that the CloudWatch Logs agent
starts at every system boot.

[ec2-user ~]$ sudo chkconfig awslogs on

Viewing CloudWatch Logs

After you have given your container instance role the proper permissions to send logs
to CloudWatch Logs, and you have configured and started the agent, your container
instance should
be sending its log data to CloudWatch Logs. You can view and search these logs in
the
AWS Management Console.

Note

New instance launches may take a few minutes to send data to CloudWatch Logs.

Choose a log stream to view. The streams are identified by the cluster name
and container instance ID that sent the logs.

Configuring CloudWatch Logs at Launch with User Data

When you launch an Amazon ECS container instance in Amazon EC2, you have the option
of passing
user data to the instance that can be used to perform common automated configuration
tasks and even run scripts after the instance starts. You can pass several types of
user
data to instances, including shell scripts, cloud-init directives, and
system services. You can also pass this data into the launch wizard as plaintext,
as a
file (this is useful for launching instances via the command line tools), or as
base64-encoded text (for API calls).

The example user data block below performs the following tasks:

Installs the awslogs package, which contains the CloudWatch Logs
agent

Installs the jq JSON query utility

Writes the configuration file for the CloudWatch Logs agent and configures the Region
to
send data to (the Region in which the container instance is located)

Gets the cluster name and container instance ID after the Amazon ECS container
agent starts and then writes those values to the CloudWatch Logs agent configuration
file
log streams

If you have created the ECS-CloudWatchLogs policy and attached it to your
ecsInstanceRole as described in CloudWatch Logs IAM Policy, then you can add the above user data block to any
container instances that you launch manually. You can also add it to an Auto Scaling
launch
configuration. Your container instances that are launched with this user data begin
sending their log data to CloudWatch Logs as soon as they launch. For more information,
see Launching an Amazon ECS Container
Instance.

Javascript is disabled or is unavailable in your browser.

To use the AWS Documentation, Javascript must be enabled. Please refer to your browser's
Help pages for instructions.