In Lambda functions, you can use log statements to send log events to CloudWatch log streams, and API Gateway automatically submits log events for requests to APIs with logging enabled.

However, it can be difficult to reconcile log events for a serverless API sent across multiple CloudWatch log groups and log streams. Tracking down logs for a specific request or tailing request logs for that operation can sometimes be a cumbersome experience.

Here are some simple logging techniques and tools that can help greatly while developing, testing, and operating your serverless API.

Technique: Capture the request ID for a particular API request

The request ID for an individual API request is always returned by API Gateway in the " x-amzn-RequestId" response header. When API Gateway logs an individual API request, the request ID is always included in the first event in the request log:

"Starting execution for request: [REQUEST_ID]"

Logs for an API Gateway API are always sent to a log group in the following format:

"API-Gateway-Execution-Logs_[API_ID]/[STAGE_NAME]"

To troubleshoot an individual API request, search for the request ID in the CloudWatch Logs console, or using the Cloudwatch API or an AWS SDK (more on tooling later).

Individual API requests are tracked independently across AWS services. Thus, an individual request to your API actually generates at least two request identifiers ("request IDs") – one for the API Gateway request and one for the Lambda invocation request.

To reduce time-to-diagnosis when analyzing logs, it is helpful to correlate both request IDs together in the logs. Log the API Gateway request ID from your Lambda function and send the API Gateway request ID ($context.requestId) to your Lambda function via a mapping template:

{
"context" : {
"request-id" : "$context.requestId"
}
…
}

Then, in your Lambda function, log the API Gateway request ID along with the Lambda request ID. The Lambda request ID is automatically included in the log message.

Using this technique allows you to quickly locate the Lambda function logs for an individual API request. Search for the request ID returned in the "x-amzn-RequestId" header in the log group for the Lambda function (by default, named "/aws/lambda/[FUNCTION_NAME]").

Tooling

While the AWS SDK and CLI provide excellent building blocks for working with API Gateway, Lambda, and CloudWatch Logs, there is still room for specialized tooling for logs when developing, testing, and operating your serverless API.

Given an API Gateway REST API ID and Stage name, this command-line tool produces an aggregated stream of time-ordered, ANSI-colored log events emitted by API Gateway and all Lambda functions attached to your API. It automatically aggregates events from all log streams for the API and Lambda functions.

For example:

Stream all log events emitted from API Gateway as well as from all Lambda functions attached to the API:

$ apilogs get --api-id xyz123 --stage prod –-watch

Search APIG/Lambda logs for events from a specific request ID in the past hour: