Filtering Event Messages

This section contains code snippets of filtering that you can implement to lessen the load of event subscription messages. To help show the differences in various languages' syntax, these snippets illustrate the same set of filters written in the following languages:

In this example, the handleRequest method, which is an AWS Lambda standard method name, takes a Map type as its first parameter, which is the event subscription message content.The second parameter it takes is the context of the current Lambda Proxy request.The Context object is used to obtain a Lambda Logger, which is used to write a message to the CloudWatch Logs.

The AWS SDK is used to invoke another Lambda, which is responsible for delivering the filtered message to our desired endpoint.The purpose of passing off the responsibility of delivering the message to another Lambda is to avoid a timeout of the delivery request coming from the Event Subscription service. Currently, the allowable timeout for delivery is set to five seconds. If the filter takes longer than allowed by the setting, you can process the request, but the Event Subscription service will time out and fall into a retry loop until it receives a 200-level response within the timeout period. To learn more about managing message delivery, see "Improving Message Delivery While Accommodating Timeouts."

Python

The main difference between the Java and Python examples is that in the Java example the event subscription message is received as the first parameter, and in the Python example the first parameter is a Lambda proxy "event," which contains the event subscription message along with information about the AWS Lambda proxy request.

The following example in Python shows how to filter project payloads based on the Group ID of the project, as done inprojectGroupFiltering.py:

Establish the group ID that you are looking for and create it as a static constant.

DESIRED_GROUP_ID = 'VaqTTVaB0UcbPu4n6824WIYYIV953Mg3'

The first parameter is the Lambda proxy "event," which contains the event subscription message and some additional information that needs to be parsed out. The second parameter is the context of the current Lambda Proxy request. In this example, the Context object is used to obtain a Lambda Logger, which is used to write a message to the CloudWatch Logs.

def project_group_filter_handler(event, context):
...

Parse out the message from the event.

event_subscription_message = json.loads(event['body'])

Obtain the “newState” attribute of the event subscription message.The newState attribute represents the updated state of the resource.

The AWS SDK is used to invoke another Lambda, which is responsible for delivering the filtered message to our desired endpoint.The purpose of passing off the responsibility of delivering the message to another Lambda is to avoid a timeout of the delivery request coming from the Event Subscription service. Currently, the timeout for delivery is set to five seconds. If the filter takes longer than allowed by the setting, you can process the request, but the Event Subscription service will time out and fall into a retry loop until it receives a 200-level response within the timeout period. To learn more about managing message delivery, see "Improving Message Delivery While Accommodating Timeouts."

Node.js

The Node.js example of project group ID filtering reads similar to the Java and Python examples. As with the Python example, the first parameter is a Lambda proxy event and the second parameter is the Lambda Context.

The following example in Node.js shows how to filter project payloads based on the Group ID of the project, as done inprojectGroupFiltering.js:

Establish the group ID that you are looking for and create it as a static constant.

const DESIRED_GROUP_ID = 'VaqTTVaB0UcbPu4n6824WIYYIV953Mg3';

The first parameter is the Lambda proxy "event," which contains the event subscription message and some additional information that needs to be parsed out. The second parameter is the context of the current Lambda Proxy request. In this example, the Context object is used to obtain a Lambda Logger, which is used to write a message to the CloudWatch Logs.

The AWS SDK is used to invoke another Lambda, which is responsible for delivering the filtered message to our desired endpoint.The purpose of passing off the responsibility of delivering the message to another Lambda is to avoid a timeout of the delivery request coming from the Event Subscription service. Currently, the timeout for delivery is set to five seconds. If the filter takes longer than allowed by the setting, you can process the request, but the Event Subscription service will time out and fall into a retry loop until it receives a 200-level response within the timeout period. To learn about managing message delivery, see "Improving Message Delivery While Accommodating Timeouts."

Improving Message Delivery While Accommodating Timeouts

The Event Subscription service has a strict timeout of five seconds for all delivery requests. In the event that the delivery of a message exceeds the allowed time, the Event Subscription service begins a retry cycle for that message.

For example, you build a project group ID filter similar to one of the examples found in "Filtering Event Messages," and you include a database lookup to determine whether the message is needed. It is possible that the database lookup along with the time needed for required processing and for the Lambda to cold-start could take more than five seconds, causing the Event Subscription service to retry delivering the message.

You can avert a retry by separating the time-consuming parts of the process from the logic that is responsible for determining whether the message is one you want to process and deliver. By doing so, you can accept the message and send back a 200-level response to the Event Subscription service, while asynchronously continuing to process or filter the message in the background (see Step 5 in "Java" for an example).

Even if your processing or filtering does not exceed the five-second timeout, it is still advantageous to separate out the first touch-point of message filtering or processing from the other processing or delivery steps on the client side. That way the handoff of the message to the destination from the Event Subscription service has minimal time and performance impact to both parties.

Implementing Hosted Filters in Cloudless Architecture

If you are unable to leverage a cloud architecture for event subscription filtering, you can still use the examples in "Filtering Event Messages" as roadmaps of how to implement your own hosted filters or processing components.

Before using the filtering examples in a cloudless environment, do the following:

Omit the Lambda-specific pieces of the examples, such as the Context parameter.

Change the invocations of other Lambdas in the examples to making additional asynchronous HTTP requests to other filters or processing components that you host.

If referring to the Python and Node.js examples, substitute the first event parameter with the first payload parameter shown in the Java example (see Step 1 in "Java").

Deploy the filters or processors with a web-based API.

Preventing Missed Event Subscription Messages

Occasionally, in a cloudless architecture, services responsible for receiving event subscription messages may be unreachable. In such an event, messages might exceed the retry limit and be lost, with no way to retrieve the information within the messages.

We recommend that during the startup of your service you implement a query asking for the most recent state of all resources that might have been included in the missed messages. As shown in the following example, you can use the filter criteria to query for resources that match that criteria and then process their current state.

By querying for resources, you ensure that your integrating systems have the most current version of resources.

Implementing Asynchronous Processing in Delivering Messages

All the examples in the "Filtering Event Messages" section pass the responsibility of delivering filtered messages to another AWS Lambda. This is done to avoid exceeding the five-second timeout in the delivery request, which is enforced by the Event Subscription service that issues the request.

In a cloudless architecture, you might need to implement an asynchronous processing mechanism similar to how the AWS SDK allows for asynchronous calls to other AWS Lambdas. Most modern programming languages have third-party or core libraries that handle asynchronous processing, allowing you to leverage the async-style of processing implemented in our examples.

Thank you for taking the time to provide feedback. We appreciate and value your contribution to our site. Feedback provided here is regularly reviewed by our Product Documentation team. Please ensure your comments are specific to improving this help article. Any questions or requests outside this help article content should be directed to our Community User Forum or by submitting a ticket to customer support.